[ofa-general] Any easy way to specify to the SM to route/zone?

[Hal Rosenstock]

On Mon, Apr 13, 2009 at 6:24 PM, Hal Rosenstock
<hal.rosenstock at gmail.com> wrote:
> On Mon, Apr 13, 2009 at 5:50 PM, Chris Worley <worleys at gmail.com> wrote:
>
> <snip...>
>
>>> Were the ports getting to LinkUp/Active before partitions were configured ?
>>
>> Yes, before I started trying to partition, all the nodes could
>> communicate... except they'd all use just one port on the server and I
>> couldn't get the throughput I needed.
>
> I suspect the switch SMA went south sometime after this.
>
> <snip...>
>
>>> So no SM initialization is occurring there since they are still just in Init.
>>
>> Correct.  But, the SM is running.
>
> Nothing the SM can do when a device SMA malfunctions like that.
>
>
> <snip...>
>
>>>> Partition "part2" with P_Key=2 should connect this client's port 0 to
>>>> the sever on port 1 of mlx4_1
>>>
>>> Do you really mean port 0 ?
>>
>> Nope... in this case I have 0x0002c903000292b0 in part2 in my
>> partitions file, which is port 1, the second port of the adapter.  I'm
>> hoping to use both ports of all adapters on the server.
>
> So you're talking about physical marking on the card rather than
> actual (logical) port number.
>
>
> <snip...>
>
>> After power-cycling the switch, the ports went "active"!  Note that I
>> didn't restart the SM... I just left it running.
>
> That should be fine.
>
>> So, on one client... the one corresponding to "part2" in the
>> partitions file, I put the P_Key into the "create child":
>>
>> echo 0x2 > /sys/class/net/ib0/create_child
>>
>> ... and did likewise on the host, for ib3 (the second port on the
>> second adapter):
>>
>> echo 0x2 > /sys/class/net/ib3/create_child
>
> I'm not 100% sure but I think you may need the full member PKey on at
> least one of them (0x800x).

Maybe the simplest thing to do is make them all full members but on
separate partitions. There are some other issues with using the
limited members I forgot about. Using full members but separate
partitions should still isolate the clients. All nodes would still
only be limited members of the default partition.

-- Hal

>> Still, no ping (the interfaces are setup correctly).
>
> Are there still join failure messages on the client and/or server ?
> What do they say now ?
>
> -- Hal
>
>> Thanks,
>>
>> Chris
>> <snip>
>