[ofa-general] question about partitioning IB networks

[Hal Rosenstock]

On Mon, Aug 31, 2009 at 3:29 PM, Meyer, Donald J
<donald.j.meyer at intel.com>wrote:

>  I am trying to partition my IB network but I don’t seem to be able to
> understand the opensm man page.
>
>
>
> First it says “The default partition has P_Key value 0x7fff. OpenSM´s port
> will have full membership in default partition. All other end ports will
> have partial membership.” but I don’t see the difference defined between
> full and partial membership anywhere.  Is it possible the reference was to
> full and limited membership instead?
>
Yes, partial == limited. I've just sent a patch to change that word in the
man page and doc.

>  Does this partition have to exist on all CA’s so the SM can “talk”
> them?
>
Yes, this is an IBA requirement.

>  Also it says the default partition will be created “unconditionally even
> when partition configuration file does not exist or cannot be accessed.”
>  Will it also be created if the partition configuration file exists but does
> not have a default partition defined?
>
No.

>   Second, I see where CA’s can be members of multiple partitions (have
> multiple P_keys).  If a CA is in multiple partitions (has multiple P_Keys
> assigned to it), which partition does it “send” on when the CA has packets
> to send if more than one partition can reach the destination CA?
>
That's up to the application/ULP to set the proper PKey index. The
application/ULP needs to ensure the destination is reachable via a common
PKey. It does that via some sort of PathRecord request to the SA.

>  Also do switches (or any non CA’s) have to have P_Keys assigned for any
> reason?
>
Yes, but with OpenSM they do not need configuration. OpenSM detects which
switches are leaf switches with peer CA ports and sets up their partition
tables appropriately.

>
>
> Just as a sanity check, my interpretation so far is that my network should
> have a partition configuration file similar to the following.  Can anyone
> tell me if I have this correct?  In this example configuration, I am trying
> to create two partitions.  One with rack one and two, the other with rack
> three and four:
>
>
>
> #Default partition (for SM control of the CA’s)
>
> Default=0x7fff,ipoib,rate=7:ALL=limited;
>
Default=0x7fff,ipoib,rate=7:ALL,SELF=full;

>   #rack1
>
> rack1=0x111,ipoib,rate=7,defmember=full:<GUID_list>;
>
> #rack2
>
> rack2=0x111,ipoib,rate=7,defmember=full:<GUID_list>;
>
> #rack3
>
> rack3=0x112,ipoib,rate=7,defmember=full:<GUID_list>;
>
> #rack4
>
> rack4=0x112,ipoib,rate=7,defmember=full:<GUID_list>;
>
I've never done it this way but it does look like the partition create code
will detect the duplicated partitions (0x111 and 0x112) and merge ports
from rack2 with rack1 and rack4 with rack3.

-- Hal


>
>
> *Thanks,*
>
> *Don Meyer*
>
> *Senior Network/System Engineer/Programmer*
>
> US+ (253) 371-9532 iNet 8-371-9532
>
> **Other names and brands may be claimed as the property of others*
>
>
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20090903/03a0827d/attachment.html>