[openib-general] RDMA CM multicast

[Sean Hefty]

>I don't this isn't as easy as you've made it sound.  I see two
>approaches to preventing address collision -- both require voluntary
>participation.  First is a centralized authority approach (this has been
>used for IP multicast-based protocols).  This means running some sort of
>daemon in a location all peers can communicate with.  I'm not really
>keen on the idea of requiring a separate daemon just to support
>multicast in Open MPI.  Second is peer-to-peer based approaches.  These
>are doable, but difficult due to numerous race conditions.  It's also
>highly desireable to minimize the time cost of joining a multicast
>group; this is especially difficult with a peer-to-peer solutions.
>
>As I've said, implementing solutions at the MPI level is doable but
>difficult.  I knew from earlier discussions that IB is able to allocate
>new, unused multicast addresses and was hoping expose that functionality
>and avoid the multicast address allocation problem.  However I hadn't
>thought of the fact that other networks supported by the RDMA CM might
>not have similar functionality.. so this might not be appropriate there.

The criteria that I would use when deciding this is how much does one technology
hijack the rdma_cm.  If a feature can be considered transport neutral, but is
only actually implemented by a specific transport, then I'm inclined to include
it.  I don't think that it's too much of a stretch to consider this feature
somewhat transport neutral, as long as we can come up with a fairly clean
implementation, which I think we can.

That said, even transport specific features, like support for SDP and the IPOIB
port space, can make sense to add into the rdma_cm because of the commonality of
the code.

>  But maybe it is worth considering how hard it is for those other
>networks to provide the functionality?

My guess is that other hardware would need to do one of the two options that you
listed.  Obviously IB chose the centralized authority approach.

- Sean