[openib-general] RFC multicast support

[Sean Hefty]

> > rdma_set_option(struct rdma_cm_id *, IPPROTO_IP, IP_ADD_MEMBERSHIP,
> > 		    struct in_addr *, sizeof(struct in_addr));
>
>I'm not sure this makes sense.  Only IB has the notion of multicast on
>QPs and only UD QPs can be used for multicast.  So why would we want a
>transport-neutral API for an IB-specific feature?

First, I'll plead ignorance.  I didn't realize that iWarp didn't support
multicast, so I'll need to rethink this some.  As for some other reasons...

The CMA API came as reasonably close as I could make it to providing socket-like
semantics.  One of the intents here is to provide an API that feels familiar to
what people are used to using.  I was trying to allow applications to define
multicast groups using IP addresses, as opposed to GIDs directly.  My
understanding is that MPI will use both connected and multicast QPs for
communication; I wanted to provide a somewhat consistent addressing model.

To answer your other post, adding this as an extension to the rdma_cm_id lets
the CMA handle device hotplug, and gives us an existing asynchronous event model
(for userspace).  The CMA might also be able to handle the QP transitions and
multicast group attach/detach calls for the user.  I'm just throwing out random
ideas now, but taking a longer term approach, the CMA interface could be
expanded to support UD QPs, to abstract SIDR and address handle management from
the user.  

>Reference counting multicast group joins is definitely something that
>IB needs.  Otherwise there's no way for userspace to use IB multicast
>in a sane way.

This is a definite requirement regardless of the resulting API.