[openib-general] multicast

[Andrew Friedley]

Sean Hefty wrote:
> I see two reasonable solutions here.  One is to provide an enumeration of RDMA
> IP addresses.  The second is to use the existing methods to enumerate all IP
> addresses, then provide a check to determine if an IP address is usable as an
> RDMA address.  The current code does the latter; although, not necessarily in an
> intuitive way.

These actually seem like the same sort of solution, the difference being 
who does the work.

> 
>>I understand RDMA CM is a generic CM intended for other types of devices
>>(ie iWARP), not just infiniband.  Will all of these devices be supported
>>under the ibverbs interface?  I'm thinking it would be a problem if
>>we're picking up interfaces that don't support ibverbs, then try to use
>>ibverbs to communicate over them.
> 
> 
> The intent is that, yes, all RDMA devices should fit under the ibverbs (to be
> renamed rdma_verbs) interface.  Changes would be made to ibverbs to accommodate
> new devices as they come along.

Very good.

> If you're using IP addresses for all RDMA devices, do you still need to exchange
> this information (which is also going over an IP address)?  Is this to get more
> control over the routes used for connections?  Do you do this even when using
> standard IP over Ethernet?

Yes and no - it's certainly not *required*, but that is how the 
abstraction has been set up.  We don't make the assumption that all 
networks in use support IP - this allows us to support networks like 
MVAPI and GM.

Yes, in a sense, it does give more control over routes.

Yes, even when TCP/IP is being used for MPI communication, as all 
networks we support are abstracted into a single API.  We then do things 
like striping, failover, and support disjoint networks (that is, we 
don't assume one peer can communicate to another via all interfaces) 
irrespective of the network in use.

Andrew