[libfabric-users] 2 questions

Biddiscombe, John A. biddisco at cscs.ch
Tue Feb 21 07:44:49 PST 2017

To answer Q1 : I discover that you have to put the dest address into the info->dest_addr fields before calling getinfo - which is a bit annoying, but after stepping though the libfabric code, I see it is using the rdma_cm functions under the hood and the rdma_get_peer_addr fails if it hasn’t been set.
(Note to self: find out how these address vectors work cos they look like they will be much easier to work with)

Q2 : I just wasn’t looking at the api properly, I can post an address and a descriptor in the fi_recv call and that’s fine.


From: Libfabric-users [mailto:libfabric-users-bounces at lists.openfabrics.org] On Behalf Of Biddiscombe, John A.
Sent: 20 February 2017 18:23
To: libfabric-users at lists.openfabrics.org
Subject: [libfabric-users] 2 questions

Question 1: I’ve created a passive endpoint (on both of two nodes) for listening, exchanged name details and created an active endpoint (on one node) that I call connect on, giving the address details of the other node where the passive endpoint is listening.

I’m getting “ret=-22 (Invalid argument)” on my call to connect – since there are only 2 arguments set in the call I’m using
            ret = fi_connect(ct.ep, name_data, NULL, 0);

dumping out the 16 bytes of buffer used for name (ip address is

enabling logging shows

libfabric:verbs:core:fi_ibv_msg_ep_connect():151<info> src_addr:
libfabric:verbs:core:fi_ibv_msg_ep_connect():158<info> dst_addr:
fi_connect(): /users/biddisco/src/hvtkm/hpx/plugins/parcelport/libfabric/libfabric_controller.hpp:667 , ret=-22 (Invalid argument)

What might be wrong? (the ping pong example is just the same, the only difference is I have a passive endpoint and an active one on the same node – I’m using the same fi_fabric/fi_domain for both –could that be what’s wrong?)

Question 2 : If I register a buffer for send/recv/read/write/etc and split it into N smaller buffers, in the post_ send/recv calls, there does not seem to be a way of using an offset into the buffer so that I can post N of them each with a different offset (c.f. verbs equivalent, which uses the same local key, but a different offset for each).
Can I do that, or do I have to register the buffer as a vetor of smaller buffers and ive each one a descriptor individually?



John Biddiscombe,                        email:biddisco @.at.@ cscs.ch
CSCS, Swiss National Supercomputing Centre  | Tel:  +41 (91) 610.82.07
Via Trevano 131, 6900 Lugano, Switzerland   | Fax:  +41 (91) 610.82.82

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20170221/f2702043/attachment.html>

More information about the Libfabric-users mailing list