[OMPI devel] [ofa-general] OpenMPI and RDMA-CM

[Or Gerlitz]

Andrew Friedley wrote:
> Jeff Squyres wrote:
>>>> FWIW, yes, adding RDMA CM support has actually been on my to-do list
>>>> for a while, but it keeps getting bumped by higher priority items.
>>>> It would be *much* better if some iWARP companies got involved in
>>>> Open MPI...

> Hmm I'm interested.  I've already done some work switching over to RDMA 
> CM for some research stuff I've been doing; it's not publicly accessible 
> w/o the 3rd party agreement.  I can help answer questions on what 
> exactly needs to change, and do some testing.

Doing a bit of zoom out from the "how to make ofed's udapl work for 
ompi" thread, my thinking is that the ompi udapl btl enablement is 
actually only the first step, where for production/longterm/etc you want 
to have an rdmacm btl. Reasoning here is made of many arguments, among 
them the quickest i can make are:

A) it seems that ompi would want to use not only RC but rather also UD 
multicast and unicast, which are not covered by udapl

B) there's actually no real justification to maintain two APIs (namely 
udapl vs libibvers/librdmacm), so down the road, only one of them would 
survive (udapl is implemented ***over*** libibverbs/librdmacm so if the 
latteres dies same does udapl). Specifically, I hear here and there that 
the OFED stack is now on its way to be deployed all over the place, 
specifically in commercial Unix OSs (which want modern! code that 
supports IPoIB-CM,RDS,SRP,iSER, etc you named it) so eventually the 
rdmacm btl can be used also over Solaris et al.

Or.