[ewg] Re: [ofa-general] how to use Intel MPI with dapl2?

[Arlin Davis]

Scott Weitzenkamp (sweitzen) wrote:
> I am using Intel MPI 3.1 build 26 with OFED 1.3 beta2 on RHEL4 x86_64.  
> Intel MPI works as before with dapl1, but I see very slow performance 
> with dapl2.  Are there extra command-line params I need to use dapl2?

Intel MPI 3.1.26 does not support uDAPL v2.

How did you configure your servers to run Intel MPI with v2 libraries? 
Did you put the ofa-v2-ib0 entry at the top of the /etc/dat.conf file or 
specify a v2 device via "-genv I_MPI_DEVICE rdssm:ofa-v2-ib0"? I didn't 
see a device option on your command line.

Anyway, uDAPL v2 support is not there yet.

>  
> $ /data/software/qa/MPI/intel_mpi/intelmpi-3.1-`uname -m`/bin/mpiexec 
> -genv I_MPI_DEBUG 3 -n 2  osu_latency.x
> *[1] MPI startup(): DAPL provider <NULL on rank 0:svbu-qa1850-1 differs 
> from <NUL
> L string> on rank 1:svbu-qa1850-2

Looks like it failed over to sockets device because of errors with the 
RDMA device specified in dat.conf. Something looks weird with your 
dat.conf configuration because Intel MPI is showing <NULL as the 
provider name (first column in dat.conf).

You can set "-genv I_MPI_FAILOVER disable" to disable failover and 
immediately bail after any RDMA device error.

-arlin