[openib-general] IB mcast question

[Steve Wise]

On Tue, 2006-08-15 at 12:39 -0700, Sean Hefty wrote:
> >Why are these separated?  Isn't an address handle needed for each
> >destination QP?  If so, then why is the remote qpn/qkey also needed to
> >transmit a datagram?
> 
> The address handle doesn't include QPN/QKey information.  Maybe think of them
> more as specifying the path to some port.
> 

Ok.

>From what I can tell via experimentation, the qkey of the mcast group
doesn't need to have any relation to the qkeys of the qps. 

I was able to create a mcast group with the mc qkey==0xe00a0a0a, and 3
apps joined this group, but their qp qkeys were 0 (I changed
ucma_init_ud_qp() to set the qp qkey to 0).  One app sent to the mcgroup
ah/qkey/qpn and the other two received the packet.  Does that make
sense?

So maybe all we need is the concept of REUSE_PORT to allow multiple
librdma users to create cm_ids with the same local port.  currently this
isn't allowed.  If we do this, then all processes that want to exchange
mcast packets would create cm_ids and do rdma_resolve_addr() with the
same src port number on all systems. 

Senders send to the ah/remote_qpn/remote_qkey of the mcast group.  This
routes packets to all IB ports that have subscribers.  Then since the
sender's qp has the same qkey as all the group participants each qp will
receive a copy of the packet.

The mcast setup code in librdma doesn't need to change.  IE the qkey can
remain the ip mcast address.

I think this will work.  It is similar to UDP/IP/MCAST...

Or am I all wet?

whatchathink?