[openib-general] [PATCH] uDAPL openib-cma provider - add support for IB_CM_REQ_OPTIONS

[Arlin Davis]

Scott Weitzenkamp (sweitzen) wrote:

>Arlin, 
>
>I'm having trouble running Intel MPI 2.0.1 and OFED 1.0 rc5 with Intel
>MPI Benchmark 2.3 on a 32-node PCI-X RHEL4 U3 i686 cluster.  This thread
>caught my eye, can you look at my output and tell me if this is the same
>issue?  If not, are there other things I can tune, or should I file a
>bug somewhere?
>
>  
>
this looks like a configuration issue and not the timeout. The CR 
timeouts occured with
the rdma device and not the rdssm.  Is IPoIB running on the ib0 
interfaces across the
fabric?

>$ .../intelmpi-2.0.1-`uname -m`/bin/mpiexec -genv I_MPI_DEBUG 3 -genv
>I_MPI_DEVICE rdssm -genv LD_LIBRARY_PATH .../intelmpi-2.0.1-`uname
>-m`/lib -n 32 .../IMB_2.3/src/IMB-MPI1 PingPong
>I_MPI: [0] set_up_devices(): will use device: libmpi.rdssm.so
>I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-cma
>I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-cma
>I_MPI: [0] set_up_devices(): will use device: libmpi.rdssm.so
>I_MPI: [0] set_up_devices(): will use DAPL provider: OpenIB-cma
>aborting job:
>Fatal error in MPI_Init: Other MPI error, error stack:
>MPIR_Init_thread(531): Initialization failed
>MPID_Init(146): channel initialization failed
>MPIDI_CH3_Init(937):
>MPIDI_CH3_Progress(328): MPIDI_CH3I_RDMA_wait_connect failed in
>VC_post_connect
>(unknown)(): (null)
>aborting job:
>Fatal error in MPI_Init: Other MPI error, error stack:
>MPIR_Init_thread(531): Initialization failed
>MPID_Init(146): channel initialization failed
>MPIDI_CH3_Init(937):
>MPIDI_CH3_Progress(328): MPIDI_CH3I_RDMA_wait_connect failed in
>VC_post_connect
>(unknown)(): (null)
>aborting job:
>Fatal error in MPI_Init: Other MPI error, error stack:
>MPIR_Init_thread(531): Initialization failed
>MPID_Init(146): channel initialization failed
>MPIDI_CH3_Init(937):
>MPIDI_CH3_Progress(328): MPIDI_CH3I_RDMA_wait_connect failed in
>VC_post_connect
>(unknown)(): (null)
>aborting job:
>Fatal error in MPI_Init: Other MPI error, error stack:
>MPIR_Init_thread(531): Initialization failed
>MPID_Init(146): channel initialization failed
>MPIDI_CH3_Init(937):
>MPIDI_CH3_Progress(328): MPIDI_CH3I_RDMA_wait_connect failed in
>VC_post_connect
>(unknown)(): (null)
>aborting job:
>Fatal error in MPI_Init: Other MPI error, error stack:
>MPIR_Init_thread(531): Initialization failed
>MPID_Init(146): channel initialization failed
>MPIDI_CH3_Init(937):
>MPIDI_CH3_Progress(328): MPIDI_CH3I_RDMA_wait_connect failed in
>VC_post_connect
>(unknown)(): (null)
>aborting job:
>Fatal error in MPI_Init: Other MPI error, error stack:
>MPIR_Init_thread(531): Initialization failed
>MPID_Init(146): channel initialization failed
>MPIDI_CH3_Init(937):
>MPIDI_CH3_Progress(328): MPIDI_CH3I_RDMA_wait_connect failed in
>VC_post_connect
>(unknown)(): (null)
>aborting job:
>
>  
>