[ofa-general] ofed1.2.5rc2 and intel mpi error

[Arlin Davis]

Mahmoud Hanafi wrote:
> 
> here is my mpirun command
> 
> mpirun -np 128 -env I_MPI_DEVICE rdma:OpenIB-cma -env I_MPI_DEBUG 2 
> /home/hanafim/HPL/xhpl
> 

What IB adapter are you using and what F/W version (ibstat)?

This might be related to following bug with large RDMA reads: 
https://bugs.openfabrics.org//show_bug.cgi?id=736

Try the following:

Set "-env I_MPI_RDMA_MAX_MSG_SIZE 4194304" to workaround the issue. If 
it does not help, set I_MPI_RDMA_RNDV_WRITE to enable to use RDMA Write 
rendezvous protocol instead of the default RDMA Read.

-arlin