[libfabric-users] Problems running fabtests

[Hefty, Sean]

> I'm having some trouble getting libfabric+verbs to work on my machine.
> 
> It's running Ubuntu 18.04 with Mellanox MT27710 CX-4 NICs. I am able to run ib_send_bw
> with both RC and UD (and other ib_* benchmarks), and get reasonable performance
> numbers.
> 
> I see both verbs and "verbs;ofi_rxm" providers if I run fi_info. fi_pingpong also runs
> fine without any flags.
> 
> However if I run fi_pingpong with the flag "-p verbs", I get the error: fi_getinfo():
> util/pingpong.c:1436, ret=-61 (No data available) on one side, and "util/pingpong.c:558
> : ctrl/read: no data or remote connection closed" on the other side.

If you have fabtests installed, you can try running fi_rdm_pingpong or fi_msg_pingpong.

I don't recall what endpoint type fi_pingpong defaults to.  But you can set this using the -e option (e.g. -e rdm or -e msg).  With verbs, you usually have to specify the source address.  I don't think fi_pingpong supports that, but it's the -s <ip_address> option for fi_rdm/msg_pingpong.

> I'm using libfabric 1.5.1, with the default mlx5_core drivers that come with Ubuntu
> 18.04. I tried installing libfabric 1.7 manually and got a different set of errors
> ("util/pingpong.c:1523: fi_av_insert: number of addresses inserted = 0; number of
> addresses given = 1") and now I'm out of ideas on how to get fabtests to work.

Did you update to fabtests v1.7 along with libfabric v1.7?  Though, fi_pingpong isn't technically a fabtests.  That ships directly with libfabric.

- Sean