[ewg] iWarp - Failed to modify RC QP to RTS

ren yufei renyufei83 at yahoo.com.cn
Sat Apr 23 13:19:00 PDT 2011


Hi Steve,

Thanks. 'rdma_bw --cma" works.

'rping' is based on RDMA CM + IB verbs, and works on IB and iWarp.
'ib_write_bw' is based on socket + IB verbs, and only works on IB.
'rdma_bw' is based on RDMA CM + IB verbs and socket + IB verbs while '--cma'.


Is that correct? If so, I am confused about RDMA CM. What's the different between the RDMA CM(rdma_connect/rdma_accept/rdma_create_qp) and soc ket plus IB(connect/accept/ibv_create_qp)?

Does that mean the app could only use RDMA CM to setup connection, then do RDMA operation in iWarp environment?

Thank you very much.
Yufei



________________________________
From: Steve Wise <swise at opengridcomputing.com>
To: ren yufei <renyufei83 at yahoo.com.cn>
Cc: OpenFabrics EWG <ewg at lists.openfabrics.org>
Sent: Friday, April 22, 2011 11:21 AM
Subject: Fw: [ewg] iWarp - Failed to modify RC QP to RTS

ib_write_bw is an Infiniband-only application.  Try rdma_bw and rdma_lat and use the --cma flag.

Steve.

On 04/21/2011 05:39 PM, ren yufei wrote:
> Hi,
>
> Two hosts connected through both IB and iWarp.
> 1 'ib_write_bw' success via IB connection, while failure via iWarp connection.
> 2 Both clinet and server export 'Failed to modify RC QP to RTS' failure.
>    2.1 client errno: 22(Invalid argument)
>    2.2 server errno: 22(Invalid argument)
> 3 'rping' works via both IB and iWarp.
>
> Any suggestion?
>
> Connection
> ==
>             client device <====>      server device
> IB         mlx4_1                                mlx4_0 (addr IB)
> iWarp    nes0                                    nes0 (addr iWarp)
>
> iWarp failure
> ==
> server# ib_write_bw -d nes0
> client# ib_write_bw [addr iWarp] -d nes0
>
> ------------------------------------------------------------------
>                     RDMA_Write BW Test
>  Number of qp's running 1
>  Connection type : RC
>  Each Qp will post up to 100 messages each time
>  Inline data is used up to 0 bytes message
>  Link type is ETH
>  Using gid index 0 as source GID
>  local address: LID 0x01 QPN 0x0062 PSN 0xf2d446 RKey 0x104e6416 VAddr 0x002b699d075000
>  GID: 00:18:85:02:248:04:00:00:00:00:00:00:00:00:00:00
>  remote address: LID 0x01 QPN 0x0277 PSN 0xbc7dd7 RKey 0x60f7b956 VAddr 0x002b6ff84f3000
>  GID: 00:18:85:02:248:136:00:00:00:00:00:00:00:00:00:00
> Failed to modify RC QP to RTS
>  Unable to Connect the HCA's through the link
>
>
> IB success
> ==
> 'ib_write_bw' via IB
> server# ib_write_bw -d mlx4_0
> client# ib_write_bw [addr IB] -d mlx4_1
>
> ------------------------------------------------------------------
>                     RDMA_Write BW Test
>  Number of qp's running 1
>  Connection type : RC
>  Each Qp will post up to 100 messages each time
>  Inline data is used up to 0 bytes message
>  Link type is ETH
>  Using gid index 0 as source GID
>  local address: LID 0000 QPN 0x30004e PSN 0x90ea46 RKey 0x10002a0f VAddr 0x002b3ceb0e9000
>  GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:218
>  remote address: LID 0000 QPN 0x18004e PSN 0x29c2f2 RKey 0x58002af9 VAddr 0x002b841d5ce000
>  GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:212
>  Mtu : 1024
> ------------------------------------------------------------------
>  #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]
> Conflicting CPU frequency values detected: 2793.000000 != 1596.000000
>  65536      5000           0.00               0.00
> ------------------------------------------------------------------
>
>
> ibv_devinfo
> ==
> client:
> hca_id: mlx4_1
>         transport:                      InfiniBand (0)
>         fw_ver:                         2.7.854
>         node_guid:                      ffff:ffff:ffff:ffff
>         sys_image_guid:                 ffff:ffff:ffff:ffff
>         vendor_id:                      0x02c9
>         vendor_part_id:                 26478
>         hw_ver:                         0xB0
>         board_id:                       MT_0D80110017
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                2048 (4)
>                         active_mtu:             1024 (3)
>                         sm_lid:                 0
>                         port_lid:               0
>                         port_lmc:               0x00
>                         link_layer:             Ethernet
>
> hca_id: nes0
>         transport:                      iWARP (1)
>         fw_ver:                         3.16
>         node_guid:                      0012:5502:f804:0000
>         sys_image_guid:                 0012:5502:f804:0000
>         vendor_id:                      0x1255
>         vendor_part_id:                 256
>         hw_ver:                         0x5
>         board_id:                       NES020 Board ID
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                4096 (5)
>                         active_mtu:             1024 (3)
>                         sm_lid:                 0
>                         port_lid:               1
>                         port_lmc:               0x00
>                         link_layer:             Ethernet
>
> server
> hca_id: mlx4_0
>         transport:                      InfiniBand (0)
>         fw_ver:                         2.7.854
>         node_guid:                      ffff:ffff:ffff:ffff
>         sys_image_guid:                 ffff:ffff:ffff:ffff
>         vendor_id:                      0x02c9
>         vendor_part_id:                 26478
>         hw_ver:                         0xB0
>         board_id:                       MT_0D80110017
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                2048 (4)
>                         active_mtu:             1024 (3)
>                         sm_lid:                 0
>                         port_lid:               0
>                         port_lmc:               0x00
>
> hca_id: nes0
>         transport:                      iWARP (1)
>         fw_ver:                         3.16
>         node_guid:                      0012:5502:f888:0000
>         sys_image_guid:                 0012:5502:f888:0000
>         vendor_id:                      0x1255
>         vendor_part_id:                 256
>         hw_ver:                         0x5
>         board_id:                       NES020 Board ID
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                4096 (5)
>                         active_mtu:             4096 (5)
>                         sm_lid:                 0
>                         port_lid:               1
>                         port_lmc:               0x00
>
> Thanks.
>
> Yufei
>
>
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20110424/7487da79/attachment.html>


More information about the ewg mailing list