<html><body><div style="color:#000; background-color:#fff; font-family:tahoma, new york, times, serif;font-size:12pt"><div>Hi,</div><div><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">Two hosts connected through both IB and iWarp.</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">1 'ib_write_bw' success via IB connection, while failure via iWarp connection.</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">2 Both clinet and server export 'Failed to modify RC QP to RTS' failure.</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"> 2.1 client errno:
22(Invalid argument)</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"> 2.2 server errno: 22(Invalid argument)<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">3 'rping' works via both IB and iWarp.<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">Any suggestion?<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family:
tahoma,new york,times,serif; background-color: transparent; font-style: normal;">Connection</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">==<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"> client device <====> server device<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">IB mlx4_1 mlx4_0
(addr IB)<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">iWarp nes0 nes0 (addr iWarp)<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">iWarp failure</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">==</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family:
tahoma,new york,times,serif; background-color: transparent; font-style: normal;">server# ib_write_bw -d nes0</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">client# ib_write_bw [addr iWarp] -d nes0</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">------------------------------------------------------------------<br> RDMA_Write BW Test<br> Number of qp's running 1<br> Connection type : RC<br> Each Qp will post up to 100 messages each time<br> Inline data is used up to 0
bytes message<br> Link type is ETH<br> Using gid index 0 as source GID<br> local address: LID 0x01 QPN 0x0062 PSN 0xf2d446 RKey 0x104e6416 VAddr 0x002b699d075000<br> GID: 00:18:85:02:248:04:00:00:00:00:00:00:00:00:00:00<br> remote address: LID 0x01 QPN 0x0277 PSN 0xbc7dd7 RKey 0x60f7b956 VAddr 0x002b6ff84f3000<br> GID: 00:18:85:02:248:136:00:00:00:00:00:00:00:00:00:00<br><span style="font-weight: bold;">Failed to modify RC QP to RTS</span><br style="font-weight: bold;"> Unable to Connect the HCA's through the link<br><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">IB success</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new
york,times,serif; background-color: transparent; font-style: normal;">==<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">'ib_write_bw' via IB</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">server# ib_write_bw -d mlx4_0</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">client# ib_write_bw [addr IB] -d mlx4_1<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style:
normal;">------------------------------------------------------------------<br> RDMA_Write BW Test<br> Number of qp's running 1<br> Connection type : RC<br> Each Qp will post up to 100 messages each time<br> Inline data is used up to 0 bytes message<br> Link type is ETH<br> Using gid index 0 as source GID<br> local address: LID 0000 QPN 0x30004e PSN 0x90ea46 RKey 0x10002a0f VAddr 0x002b3ceb0e9000<br> GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:218<br> remote address: LID 0000 QPN 0x18004e PSN 0x29c2f2 RKey 0x58002af9 VAddr 0x002b841d5ce000<br> GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:212<br> Mtu : 1024<br>------------------------------------------------------------------<br> #bytes #iterations BW
peak[MB/sec] BW average[MB/sec]<br>Conflicting CPU frequency values detected: 2793.000000 != 1596.000000<br> 65536 5000 0.00 0.00<br>------------------------------------------------------------------<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">ibv_devinfo</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif;
background-color: transparent; font-style: normal;">==</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">client:</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">hca_id: mlx4_1<br> transport: InfiniBand (0)<br> fw_ver: 2.7.854<br> node_guid:
ffff:ffff:ffff:ffff<br> sys_image_guid: ffff:ffff:ffff:ffff<br> vendor_id: 0x02c9<br> vendor_part_id: 26478<br> hw_ver: 0xB0<br> board_id:
MT_0D80110017<br> phys_port_cnt: 1<br> port: 1<br> state: PORT_ACTIVE (4)<br> max_mtu: 2048 (4)<br>
active_mtu: 1024 (3)<br> sm_lid: 0<br> port_lid: 0<br> port_lmc: 0x00<br>
link_layer: Ethernet<br><br>hca_id: nes0<br> transport: iWARP (1)<br> fw_ver: 3.16<br> node_guid: 0012:5502:f804:0000<br> sys_image_guid: 0012:5502:f804:0000<br>
vendor_id: 0x1255<br> vendor_part_id: 256<br> hw_ver: 0x5<br> board_id: NES020 Board ID<br> phys_port_cnt:
1<br> port: 1<br> state: PORT_ACTIVE (4)<br> max_mtu: 4096 (5)<br> active_mtu: 1024
(3)<br> sm_lid: 0<br> port_lid: 1<br> port_lmc: 0x00<br> link_layer:
Ethernet<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">server</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">hca_id: mlx4_0<br> transport: InfiniBand (0)<br> fw_ver: 2.7.854<br>
node_guid: ffff:ffff:ffff:ffff<br> sys_image_guid: ffff:ffff:ffff:ffff<br> vendor_id: 0x02c9<br> vendor_part_id: 26478<br> hw_ver: 0xB0<br>
board_id: MT_0D80110017<br> phys_port_cnt: 1<br> port: 1<br> state: PORT_ACTIVE (4)<br> max_mtu: 2048
(4)<br> active_mtu: 1024 (3)<br> sm_lid: 0<br> port_lid: 0<br> port_lmc:
0x00<br><br>hca_id: nes0<br> transport: iWARP (1)<br> fw_ver: 3.16<br> node_guid: 0012:5502:f888:0000<br> sys_image_guid: 0012:5502:f888:0000<br>
vendor_id: 0x1255<br> vendor_part_id: 256<br> hw_ver: 0x5<br> board_id: NES020 Board ID<br> phys_port_cnt:
1<br> port: 1<br> state: PORT_ACTIVE (4)<br> max_mtu: 4096 (5)<br> active_mtu: 4096
(5)<br> sm_lid: 0<br> port_lid: 1<br> port_lmc: 0x00<br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new
york,times,serif; background-color: transparent; font-style: normal;">Thanks.</div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 20px; font-family: tahoma,new york,times,serif; background-color: transparent; font-style: normal;">Yufei<br></div></div></body></html>