[ewg] iWarp - Failed to modify RC QP to RTS

Steve Wise swise at opengridcomputing.com
Fri Apr 22 08:21:30 PDT 2011


ib_write_bw is an Infiniband-only application.  Try rdma_bw and rdma_lat and use the --cma flag.

Steve.

On 04/21/2011 05:39 PM, ren yufei wrote:
> Hi,
>
> Two hosts connected through both IB and iWarp.
> 1 'ib_write_bw' success via IB connection, while failure via iWarp connection.
> 2 Both clinet and server export 'Failed to modify RC QP to RTS' failure.
>    2.1 client errno: 22(Invalid argument)
>    2.2 server errno: 22(Invalid argument)
> 3 'rping' works via both IB and iWarp.
>
> Any suggestion?
>
> Connection
> ==
>             client device <====>      server device
> IB         mlx4_1                                mlx4_0 (addr IB)
> iWarp    nes0                                    nes0 (addr iWarp)
>
> iWarp failure
> ==
> server# ib_write_bw -d nes0
> client# ib_write_bw [addr iWarp] -d nes0
>
> ------------------------------------------------------------------
>                     RDMA_Write BW Test
>  Number of qp's running 1
>  Connection type : RC
>  Each Qp will post up to 100 messages each time
>  Inline data is used up to 0 bytes message
>  Link type is ETH
>  Using gid index 0 as source GID
>  local address: LID 0x01 QPN 0x0062 PSN 0xf2d446 RKey 0x104e6416 VAddr 0x002b699d075000
>  GID: 00:18:85:02:248:04:00:00:00:00:00:00:00:00:00:00
>  remote address: LID 0x01 QPN 0x0277 PSN 0xbc7dd7 RKey 0x60f7b956 VAddr 0x002b6ff84f3000
>  GID: 00:18:85:02:248:136:00:00:00:00:00:00:00:00:00:00
> Failed to modify RC QP to RTS
>  Unable to Connect the HCA's through the link
>
>
> IB success
> ==
> 'ib_write_bw' via IB
> server# ib_write_bw -d mlx4_0
> client# ib_write_bw [addr IB] -d mlx4_1
>
> ------------------------------------------------------------------
>                     RDMA_Write BW Test
>  Number of qp's running 1
>  Connection type : RC
>  Each Qp will post up to 100 messages each time
>  Inline data is used up to 0 bytes message
>  Link type is ETH
>  Using gid index 0 as source GID
>  local address: LID 0000 QPN 0x30004e PSN 0x90ea46 RKey 0x10002a0f VAddr 0x002b3ceb0e9000
>  GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:218
>  remote address: LID 0000 QPN 0x18004e PSN 0x29c2f2 RKey 0x58002af9 VAddr 0x002b841d5ce000
>  GID: 254:128:00:00:00:00:00:00:02:02:201:255:254:08:254:212
>  Mtu : 1024
> ------------------------------------------------------------------
>  #bytes     #iterations    BW peak[MB/sec]    BW average[MB/sec]
> Conflicting CPU frequency values detected: 2793.000000 != 1596.000000
>  65536      5000           0.00               0.00
> ------------------------------------------------------------------
>
>
> ibv_devinfo
> ==
> client:
> hca_id: mlx4_1
>         transport:                      InfiniBand (0)
>         fw_ver:                         2.7.854
>         node_guid:                      ffff:ffff:ffff:ffff
>         sys_image_guid:                 ffff:ffff:ffff:ffff
>         vendor_id:                      0x02c9
>         vendor_part_id:                 26478
>         hw_ver:                         0xB0
>         board_id:                       MT_0D80110017
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                2048 (4)
>                         active_mtu:             1024 (3)
>                         sm_lid:                 0
>                         port_lid:               0
>                         port_lmc:               0x00
>                         link_layer:             Ethernet
>
> hca_id: nes0
>         transport:                      iWARP (1)
>         fw_ver:                         3.16
>         node_guid:                      0012:5502:f804:0000
>         sys_image_guid:                 0012:5502:f804:0000
>         vendor_id:                      0x1255
>         vendor_part_id:                 256
>         hw_ver:                         0x5
>         board_id:                       NES020 Board ID
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                4096 (5)
>                         active_mtu:             1024 (3)
>                         sm_lid:                 0
>                         port_lid:               1
>                         port_lmc:               0x00
>                         link_layer:             Ethernet
>
> server
> hca_id: mlx4_0
>         transport:                      InfiniBand (0)
>         fw_ver:                         2.7.854
>         node_guid:                      ffff:ffff:ffff:ffff
>         sys_image_guid:                 ffff:ffff:ffff:ffff
>         vendor_id:                      0x02c9
>         vendor_part_id:                 26478
>         hw_ver:                         0xB0
>         board_id:                       MT_0D80110017
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                2048 (4)
>                         active_mtu:             1024 (3)
>                         sm_lid:                 0
>                         port_lid:               0
>                         port_lmc:               0x00
>
> hca_id: nes0
>         transport:                      iWARP (1)
>         fw_ver:                         3.16
>         node_guid:                      0012:5502:f888:0000
>         sys_image_guid:                 0012:5502:f888:0000
>         vendor_id:                      0x1255
>         vendor_part_id:                 256
>         hw_ver:                         0x5
>         board_id:                       NES020 Board ID
>         phys_port_cnt:                  1
>                 port:   1
>                         state:                  PORT_ACTIVE (4)
>                         max_mtu:                4096 (5)
>                         active_mtu:             4096 (5)
>                         sm_lid:                 0
>                         port_lid:               1
>                         port_lmc:               0x00
>
> Thanks.
>
> Yufei
>
>
> _______________________________________________
> ewg mailing list
> ewg at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg




More information about the ewg mailing list