[openib-general] ipoib & the partial pkey

[Or Gerlitz]

Sean Hefty wrote:
> I looked into this more...
> RFC 4391 states (middle of page 5):
> For a node to join a partition, one of its ports must be assigned the relevant
> P_Key by the SM [RFC4392].

> Jumping to RFC 4392 (top of page 4):

Just to have us agree on the quote, it is from section 4 of rfc 4392 
(page 14) eg in http://www.ietf.org/rfc/rfc4392.txt

> at the time of creating an IB multicast group, multiple values such as the
> P_Key, Q_Key, Service Level, Hop Limit, Flow ID, TClass, MTU, etc.  have to be
> specified.  These values should be such that all potential members of the IB
> multicast group are able to communicate with one another when using them.

OK, I suggest to remove this spec limitation, as it does not allow the 
use case of a server using a partition for which inter-client 
communication is not allowed.

Actually since it does not let people use partial membership 
partitioning with IPoIB as every ipoib device needs to join the 
broadcast group, it is probably a spec bug and not a limitation done on 
purpose.

A simple real-life example is I/O target, the system admin wants IB 
block and/or file storage traffic to use a partition, but he does not 
want initiators to communicate among themselves on this partition.

To achieve that the SM is configured to assign the partial pkey to the 
initiator nodes and the full pkey to the target ports.

The current implementation of IPoIB and core perfectly (and 
transparently...) supports that.

Or.