[ewg] iWarp - Failed to modify RC QP to RTS
ren yufei
renyufei83 at yahoo.com.cn
Sat Apr 23 13:19:00 PDT 2011
Hi Steve,
Thanks. 'rdma_bw --cma" works.
'rping' is based on RDMA CM + IB verbs, and works on IB and iWarp.
'ib_write_bw' is based on socket + IB verbs, and only works on IB.
'rdma_bw' is based on RDMA CM + IB verbs and socket + IB verbs while '--cma'.
Is that correct? If so, I am confused about RDMA CM. What's the different between the RDMA CM(rdma_connect/rdma_accept/rdma_create_qp) and soc ket plus IB(connect/accept/ibv_create_qp)?
Does that mean the app could only use RDMA CM to setup connection, then do RDMA operation in iWarp environment?
Thank you very much.
Yufei
________________________________
From: Steve Wise <swise at opengridcomputing.com>
To: ren yufei <renyufei83 at yahoo.com.cn>
Cc: OpenFabrics EWG <ewg at lists.openfabrics.org>
Sent: Friday, April 22, 2011 11:21 AM
Subject: Fw: [ewg] iWarp - Failed to modify RC QP to RTS
ib_write_bw is an Infiniband-only application. Try rdma_bw and rdma_lat and use the --cma flag.
Steve.
On 04/21/2011 05:39 PM, ren yufei wrote:
> Hi,
>
> Two hosts connected through both IB and iWarp.
> 1 'ib_write_bw' success via IB connection, while failure via iWarp connection.
> 2 Both clinet and server export 'Failed to modify RC QP to RTS' failure.
> 2.1 client errno: 22(Invalid argument)
> 2.2 server errno: 22(Invalid argument)
> 3 'rping' works via both IB and iWarp.
>
> Any suggestion?
>
> Connection
> ==
> client device <====> server device
> IB mlx4_1 mlx4_0 (addr IB)
> iWarp nes0 nes0 (addr iWarp)
>
> iWarp failure
> ==
> server# ib_write_bw -d nes0
> client# ib_write_bw [addr iWarp] -d nes0
>
> ------------------------------------------------------------------
> RDMA_Write BW Test
> Number of qp's running 1
> Connection type : RC
> Each Qp will post up to 100 messages each time
> Inline data is used up to 0 bytes message
> Link type is ETH
> Using gid index 0 as source GID
> local address: LID 0x01 QPN 0x0062 PSN 0xf2d446 RKey 0x104e6416 VAddr 0x002b699d075000
> GID: 00:18:85:02:248:04:00:00:00:00:00:00:00:00:00:00
> remote address: LID 0x01 QPN 0x0277 PSN 0xbc7dd7 RKey 0x60f7b956 VAddr 0x002b6ff84f3000
> GID: 00:18:85:02:248:136:00:00:00:00:00:00:00:00:00:00
> Failed to modify RC QP to RTS
> Unable to Connect the HCA's through the link
>
>
> IB success
> ==
> 'ib_write_bw' via IB
> server# ib_write_bw -d mlx4_0
> client# ib_write_bw [addr IB] -d mlx4_1
>
> ------------------------------------------------------------------
> RDMA_Write BW Test
> Number of qp's running 1
> Connection type : RC
> Each Qp will post up to 100 messages each time
> Inline data is used up to 0 bytes message
> Link type is ETH
> Using gid index 0 as source GID
> local address: LID 0000 QPN 0x30004e PSN 0x90ea46 RKey 0x10002a0f VAddr 0x002b3ceb0e9000
> GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:218
> remote address: LID 0000 QPN 0x18004e PSN 0x29c2f2 RKey 0x58002af9 VAddr 0x002b841d5ce000
> GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:212
> Mtu : 1024
> ------------------------------------------------------------------
> #bytes #iterations BW peak[MB/sec] BW average[MB/sec]
> Conflicting CPU frequency values detected: 2793.000000 != 1596.000000
> 65536 5000 0.00 0.00
> ------------------------------------------------------------------
>
>
> ibv_devinfo
> ==
> client:
> hca_id: mlx4_1
> transport: InfiniBand (0)
> fw_ver: 2.7.854
> node_guid: ffff:ffff:ffff:ffff
> sys_image_guid: ffff:ffff:ffff:ffff
> vendor_id: 0x02c9
> vendor_part_id: 26478
> hw_ver: 0xB0
> board_id: MT_0D80110017
> phys_port_cnt: 1
> port: 1
> state: PORT_ACTIVE (4)
> max_mtu: 2048 (4)
> active_mtu: 1024 (3)
> sm_lid: 0
> port_lid: 0
> port_lmc: 0x00
> link_layer: Ethernet
>
> hca_id: nes0
> transport: iWARP (1)
> fw_ver: 3.16
> node_guid: 0012:5502:f804:0000
> sys_image_guid: 0012:5502:f804:0000
> vendor_id: 0x1255
> vendor_part_id: 256
> hw_ver: 0x5
> board_id: NES020 Board ID
> phys_port_cnt: 1
> port: 1
> state: PORT_ACTIVE (4)
> max_mtu: 4096 (5)
> active_mtu: 1024 (3)
> sm_lid: 0
> port_lid: 1
> port_lmc: 0x00
> link_layer: Ethernet
>
> server
> hca_id: mlx4_0
> transport: InfiniBand (0)
> fw_ver: 2.7.854
> node_guid: ffff:ffff:ffff:ffff
> sys_image_guid: ffff:ffff:ffff:ffff
> vendor_id: 0x02c9
> vendor_part_id: 26478
> hw_ver: 0xB0
> board_id: MT_0D80110017
> phys_port_cnt: 1
> port: 1
> state: PORT_ACTIVE (4)
> max_mtu: 2048 (4)
> active_mtu: 1024 (3)
> sm_lid: 0
> port_lid: 0
> port_lmc: 0x00
>
> hca_id: nes0
> transport: iWARP (1)
> fw_ver: 3.16
> node_guid: 0012:5502:f888:0000
> sys_image_guid: 0012:5502:f888:0000
> vendor_id: 0x1255
> vendor_part_id: 256
> hw_ver: 0x5
> board_id: NES020 Board ID
> phys_port_cnt: 1
> port: 1
> state: PORT_ACTIVE (4)
> max_mtu: 4096 (5)
> active_mtu: 4096 (5)
> sm_lid: 0
> port_lid: 1
> port_lmc: 0x00
>
> Thanks.
>
> Yufei
>
>
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20110424/7487da79/attachment.html>
More information about the ewg
mailing list