[openib-general] multicast code/merge status

[Or Gerlitz]

Sean Hefty wrote:
>> mimic IPoIB qkey flow:

>> +3 on rdma_create_qp do modify qp with some def qkey (eg zero)
>> +4 on the join completion path before attaching a qp to the associated
>> mgid, do modify qp with this mrec qkey (=ipv4 broadcast one)

> The rdma cm allows UD QP communication, which requires a valid qkey 
> before or without joining a multicast group.  I'd like to find a way to 
> continue to support this.

sure, it can use the rdmacm qkey (0x1234567 etc) when it creates the QP 
and later --if-- the user joins a multicast group modify the qp state 
with the group qkey and report it in the cma event such that the 
consumer of the rdmacm would set this into his IB UD TX WR

>> +3 on rdma_create_qp do modify qp with some def qkey (eg zero)
>> +4 on the join completion path before attaching a qp to the associated
>> mgid, do modify qp with this mrec qkey (=ipv4 broadcast one)

> Isn't the ipoib qkey a privileged qkey?

looking in ipoib code you can see the following code in 
ipoib_mcast_join_task

> 	if (!test_bit(IPOIB_MCAST_FLAG_ATTACHED, &priv->broadcast->flags)) {
> 		ipoib_mcast_join(dev, priv->broadcast, 0);
> 		return;
> 	}

so ipoib_mcast_join is called with create=0 for the broadcast group and 
this makes
it provide a component mask of

> 	comp_mask =
> 		IB_SA_MCMEMBER_REC_MGID         |
> 		IB_SA_MCMEMBER_REC_PORT_GID     |
> 		IB_SA_MCMEMBER_REC_PKEY         |
> 		IB_SA_MCMEMBER_REC_JOIN_STATE;

that is the SA sets the QKEY, RATE, MTU, SL etc etc for the broadcast 
group and later other
any joins done by ipoib uses the priv->broadcast->mcmember fields

So the broadcast qkey is basically what the SA has set when it created 
the group.

During my talking here i got a pointer to section 10 in the IPoIB RFC 
(4391)
mentioning something like "some 3rd party --has-- to create the 
broadcast group":

> 10.  Sending and Receiving IP Multicast Packets

> A node joining an IP multicast group must first construct an MGID
>    according to the rule described in section 4 above.  Once the correct
>    MGID is calculated, the node must call the SA of the outbound link to
>    attempt a "FullMember" join of the IB multicast group corresponding
>    to the MGID.  If the IB multicast group does not already exist, one
>    must be created first with the IPoIB link MTU.  The MGID MUST use the
>    same P_Key, Q_Key, SL, MTU, and HopLimit as those used in the
>    broadcast-GID.  The rest of attributes SHOULD follow the values used
>    in the broadcast-GID as well.

Bottom line, Looking in the IB SPEC and IPoIB RFC i did not see 
mentioning of privileged QKEY.

Or.