[ofa-general] Re: Using RDMA CM with MPI

[Or Gerlitz]

Keshetti Mahesh wrote:
>    I want to use the RDMA CM option of MVAPICH2. The procedure described
>    in the user guide is not much informative. ... Also, I'll be glad if some one can give
>    me a document describing how it works in detail.
try looking on the rdma_cm(7) man page installed by librdmacm
>    Actually I have some doubts like, how the IP addresses (???) are
>    resolved into IB   addresses and what happens in the case of nodes two HCAs
>    (or 1 HCA with two ports) ?
each port you want to use for your job (HCA with two ports, two HCAs, 
etc) has to have an IP address associated with it. This IP address (or a 
host name which is translated to it through DNS lookup) is probably what 
you want the rank to advertise. RDMA address resolution uses route 
lookup and ARP to learn the local <HCA, port, PKEY> and remote GID.
>
>    In the MVAPICH2 user guide it is mentioned that "RDMA CM device needs
>    to be setup, configured with an IP address and connected to the network".
>    Is this same as configuring IPoIB device ?
for IB, yes.

Or.