<html><body bgcolor="#FFFFFF"><div>oops forwarding to the right list. Any help with this issue would be appreciated.</div><div><br></div><div>Thanks</div><div><br></div><div></div><blockquote type="cite"><div>Hello,<br><br>I am running into the following issue while trying to run osu_latency:<br><br>--<br>-bash-3.2$ mpiexec --mca btl openib,self -mca btl_openib_warn_default_gid_<div id=":yr">prefix 0 -np 2 --hostfile mpihosts /home/jagga/osu-micro-benchmarks-3.3/openmpi/ofed-1.5.2/bin/osu_latency<br>
# OSU MPI Latency Test v3.3<br># Size Latency (us)<br>[amber04][[10252,1],1][connect/btl_openib_connect_oob.c:325:qp_connect_all] error modifing QP to RTR errno says Invalid argument<br>[amber04][[10252,1],1][connect/btl_openib_connect_oob.c:815:rml_recv_cb] error in endpoint reply start connect<br>
--------------------------------------------------------------------------<br>mpiexec has exited due to process rank 1 with PID 6781 on<br>node amber04 exiting without calling "finalize". This may<br>have caused other processes in the application to be<br>
terminated by signals sent by mpiexec (as reported here).<br>--------------------------------------------------------------------------<br>--<br><br>I can get around this by adding the "--mca btl_openib_cpc_include rdmacm" option. However, I have another host with a different HCA with all the same drivers and software versions that I can run this same command successfully with using the rdmacm option. What could be causing one of my environments to fail but the other to work fine (without the rdmacm option)? <br>
<br>--<br>[root@amber03 ~]# ofed_info | grep OFED<br>MLNX_OFED_LINUX-1.5.2-1.0.0 (OFED-1.5.2-20101020-1520):<br>MLNX_OFED_LINUX-1.5.2-1.0.0 (/mswg/release/ofed-1.5.2-rpms/rnfs-utils/rnfs-utils-1.1.5-10.OFED.src.rpm):<br><br>
[root@amber03 ~]# ibv_devinfo <br>hca_id: mlx4_0<br> transport: InfiniBand (0)<br> fw_ver: 2.7.9294<br> node_guid: 78e7:d103:0021:8884<br> sys_image_guid: 78e7:d103:0021:8887<br>
vendor_id: 0x02c9<br> vendor_part_id: 26438<br> hw_ver: 0xB0<br> board_id: HP_0200000003<br> phys_port_cnt: 2<br> port: 1<br> state: PORT_ACTIVE (4)<br>
max_mtu: 2048 (4)<br> active_mtu: 2048 (4)<br> sm_lid: 1<br> port_lid: 20<br> port_lmc: 0x00<br> link_layer: IB<br>
<br> port: 2<br> state: PORT_ACTIVE (4)<br> max_mtu: 2048 (4)<br> active_mtu: 1024 (3)<br> sm_lid: 0<br> port_lid: 0<br>
port_lmc: 0x00<br> link_layer: Ethernet<br>--<br><br>Any help would be greatly appreciated.<br><br>Thanks,<br><font color="#888888">-J</font></div>
</div></blockquote></body></html>