[ofa-general] Re: [mvapich-discuss] Using RDMA CM with MVAPICH2

[Sundeep Narravula]

Hi Mahesh,

Thanks for trying out our RDMA CM support. My answers are inline.

> I'vecouple of more questions to ask you.
>
> Below are the steps mentioned in MVAPICH user guide for
> running MPI application with RDMA CM support.
>
> *• Setup the RDMA CM device: RDMA CM device needs to be
> setup, configured with an IP address and connected to the network.*
>
> I have two machines (n0 and n1)connected with one ethernet interface
> and 2 IB interfaces in each. And /etc/hosts on both machines is like
> below.
>
> 192.168.3.1   n0
> 192.168.3.2   n1
> 172.131.15.1   n0_ib0
> 172.131.15.2   n0_ib1
> 172.131.15.3   n1_ib0
> 172.131.15.4   n1_ib0
>
> Now If I want to run an MPI job on both of the nodes what should I
> mention in the *'hostfile'* given to MPI ("n0, n1"  or "n0_ib0, n1_ib0 ... "
> ) ?

You can use any one of these pairs in your hostfile. i.e. using n0 and n1
should work fine.

> *• Setup the Local Address File: Create the file (/etc/mv2.conf) with the
> local IP address to be used by RDMA CM.
> $ echo 10.1.1.1 >> /etc/mv2.conf*
>
> Why is this file (/etc/mv2.conf) required ? Is it required to be present on
> all nodes?

The local rdma-cm device that the mpi library needs to use is specified in
the /etc/mv2.conf file. The file needs to be on all the machines.

  --Sundeep.

>
> -Mahesh
>