[openib-general] ipoib & the partial pkey

[Hal Rosenstock]

On Sun, 2007-02-25 at 05:48, Or Gerlitz wrote:
> Sean Hefty wrote:
> > I looked into this more...
> > RFC 4391 states (middle of page 5):
> > For a node to join a partition, one of its ports must be assigned the relevant
> > P_Key by the SM [RFC4392].
> 
> > Jumping to RFC 4392 (top of page 4):
> 
> Just to have us agree on the quote, it is from section 4 of rfc 4392 
> (page 14) eg in http://www.ietf.org/rfc/rfc4392.txt
> 
> > at the time of creating an IB multicast group, multiple values such as the
> > P_Key, Q_Key, Service Level, Hop Limit, Flow ID, TClass, MTU, etc.  have to be
> > specified.  These values should be such that all potential members of the IB
> > multicast group are able to communicate with one another when using them.
> 
> OK, I suggest to remove this spec limitation,

IMO you would need to get the IB spec changed first in order to do this.

> as it does not allow the 
> use case of a server using a partition for which inter-client 
> communication is not allowed.

> Actually since it does not let people use partial membership 
> partitioning with IPoIB as every ipoib device needs to join the 
> broadcast group, it is probably a spec bug and not a limitation done on 
> purpose.

I'm pretty sure this was done on purpose (a conscious choice) as it is
based on what the IBA spec requires.

The flip side of this approach are the partial connectivity issues which
Sean mentioned and this will be reported as SM failures (e.g. more
support issues).

> A simple real-life example is I/O target, the system admin wants IB 
> block and/or file storage traffic to use a partition, but he does not 
> want initiators to communicate among themselves on this partition.
> 
> To achieve that the SM is configured to assign the partial pkey to the 
> initiator nodes and the full pkey to the target ports.
> 
> The current implementation of IPoIB and core perfectly (and 
> transparently...) supports that.

and is currently non compliant in its behavior.

-- Hal

> Or.
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>