[openib-general] ipoib & the partial pkey, was: librdmacm: fix bug causing failure to work with partial membership pkey

[Sean Hefty]

>Doesn't this allow ipoib to join a multicast group for which it may not be able
>to communicate with all members?  For the broadcast group, this seems like an
>error to me.  Can ipoib work in such a configuration?  If all nodes were
>assigned a partial membership PKey, none of them could communicate, but no
>errors would be generated anywhere.

I looked into this more...

RFC 4391 states (middle of page 5):

For a node to join a partition, one of its ports must be assigned the relevant
P_Key by the SM [RFC4392].

Jumping to RFC 4392 (top of page 4):

at the time of creating an IB multicast group, multiple values such as the
P_Key, Q_Key, Service Level, Hop Limit, Flow ID, TClass, MTU, etc.  have to be
specified.  These values should be such that all potential members of the IB
multicast group are able to communicate with one another when using them.

and page 14:

Note that this IB_join to the broadcast group is a FullMember join.  If any of
the ports or the switches linking the port to the rest of the IPoIB subnet
cannot support the parameters (e.g., path MTU or P_Key) associated with the
broadcast group, then the IB_join request will fail and the requesting port will
not become part of the IPoIB subnet

My initial interpretation of these statements lead me to believe that pkey check
in ib_find_cached_pkey should not mask out the upper bit, which would prevent
ipoib from joining a multicast group until it has been configured with the full
membership pkey for the broadcast group.  Does this seem reasonable?

- Sean