[Users] Issues with RDMA tests on CentOS 7.3

[Susinthiran Sithamparanathan]

Hi all,
Recently i came over qferf as i'm looking for RDMA benchmark/test tools to
test/measure/study RoCE using 2 Mellanox ConnectX-3 (MCX354A-FCBT) that are
connected back-to-back on 2 physical servers. Using the following OFED
stack:
[root at compute7 qperf]# ofed_info -s
*MLNX_OFED_LINUX-4.1-1.0.2.0:*

In my attempt to run qperf on my systems, i encounter the following issues:

[root at compute8 scripts]# qperf -vv  compute7 rc_rdma_write_bw
rc_rdma_write_bw:
*failed to modify QP to RTR: Network is unreachable*
*server: failed to modify QP to RTR: Network is unreachable*
[root at compute8 scripts]# qperf -vv  compute7 uc_lat
uc_lat:
*failed to modify QP to RTR: Network is unreachable*
[root at compute8 scripts]# qperf -vv  compute7 tcp_lat
tcp_lat:
    latency          =    45.8 us
    msg_rate         =    21.8 K/sec
    msg_size         =       1 bytes
    time             =       2 sec
    timeout          =       5 sec
    loc_cpus_used    =      10 % cpus
    loc_cpus_user    =     0.5 % cpus
    loc_cpus_intr    =       2 % cpus
    loc_cpus_kernel  =     7.5 % cpus
    loc_real_time    =       2 sec
    loc_cpu_time     =     200 ms
    loc_send_bytes   =    21.8 KB
    loc_recv_bytes   =    21.8 KB
    loc_send_msgs    =  21,815
    loc_recv_msgs    =  21,814
    rem_cpus_used    =      10 % cpus
    rem_cpus_user    =     0.5 % cpus
    rem_cpus_intr    =       1 % cpus
    rem_cpus_kernel  =     8.5 % cpus
    rem_real_time    =       2 sec
    rem_cpu_time     =     200 ms
    rem_send_bytes   =    21.8 KB
    rem_recv_bytes   =    21.8 KB
    rem_send_msgs    =  21,815
    rem_recv_msgs    =  21,815
[root at compute8 scripts]# qperf -t 10 --use_bits_per_sec  compute7 tcp_bw
tcp_bw:
    bw  =  9.14 Gb/sec

I.e none of the RDMA related tests work, all come with errors. The version
of qperf is 4.9 that's shipping with CentOS 7.3. I've also built the recent
version, 4.10, from github and i experience the same issue. During the
build, i noticed the configure script detected the RDMA libs so i do
believe that's not the issue.

Am i doing something wrong? Can someone shed some lights?


Thanks!



-- 

Susinthiran Sithamparanathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20171119/52838071/attachment.html>