[libfabric-users] rdma fail with verbs

Ilango, Arun arun.ilango at intel.com
Fri Mar 3 14:40:01 PST 2017

Hi John,

Did you try running it with a stable version of libfabric (1.4.1)? Is there a reason you want to be on the tip of the master? There were a few libfabric API changes and updates to verbs provider that might have broken the rma_bw test. I’m gonna look into it now.

Btw, verbs supports RDM with some limitations. You may want to look at the fi_verbs man page to see what is supported.

>Is there a test that does rdma and is known to work with verbs?

On the master, the fi_msg_rma test works fine. Please try that if you want to test RDMA on verbs.


From: Libfabric-users [mailto:libfabric-users-bounces at lists.openfabrics.org] On Behalf Of Biddiscombe, John A.
Sent: Friday, March 03, 2017 2:26 PM
To: libfabric-users at lists.openfabrics.org
Subject: [libfabric-users] rdma fail with verbs

I’m still having trouble with rdma read. I’m using (for now) endpoint type MSG on verbs provider and I found that calling fi_read(….) returns

00: <ERROR> 0x00002aaab5626f20 0x2aaab5200700 fi_read error : Function not implemented

I have tried everything I can to adjust params/caps/hints/flags, but it seems that the verbs provider doesn’t support rdma.

I considered using endpoint type RDM, but unfortunately, RDM is not supported by verbs either, and ep MSG not working any more with GNI (I only get “connection rejected” messages when I try it), so the two providers I’m targeting are mutually incompatible – this is making development decisions difficult.

I’ve been going through the fabtests looking for one that will do RDMA and works, but using the verbs provider always fails

~/apps/fabtests/bin/fi_rma_bw -e msg -p verbs greina8
fi_getinfo(): common/shared.c:521, ret=-61 (No data available)
~/apps/fabtests/bin/fi_rma_bw -e rdm -d mlx5_0
fi_getinfo(): common/shared.c:521, ret=-61 (No data available)

only results in error.

Is there a test that does rdma and is known to work with verbs?


I updated both fabtests and libfabric from git master today or yesterday. (I’m sure endpoint MSG type was working on GNI previously, but it isn’t working for me now, should I go back to a fixed release sha in git?)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20170303/945f90ff/attachment.html>

More information about the Libfabric-users mailing list