[libfabric-users] FI_EP_DGRAM using fi_verbs provider

[Xiong, Jianxin]

Hi Anders,

The FI_ADDR_IB_UD address format has a provider defined layout that you may not be able to construct unless you dig info the provider source which is not recommended. There are two ways to get the address to be used for fi_av_insert:

  1.  Create endpoint and use fi_getname() to get the local address (endpoint name is the address) and then use an out-of-band mechanism (e.g. a socket connection, or PMI) to exchange the address. This method doesn’t use the name server.
  2.  Pass non-NULL “node” and “service” of the remote side to the fi_getinfo() call and the resulting “info” structure should have the destination address set in “info->dest_addr”. This method requires the use of the name server.
The name server is used to resolve “node + service” into an address of format FI_ADDR_IB_UD, where “node” can be a host name or a host IP address (the string, not the binary format) and “service” is usually the port number (also the string). It is only used by fi_getinfo(). IP address can’t be used as input for fi_av_insert().

Best regards,
Jianxin

From: Libfabric-users <libfabric-users-bounces at lists.openfabrics.org> On Behalf Of Anders Dam Jensen
Sent: Sunday, August 11, 2024 2:22 AM
To: libfabric-users at lists.openfabrics.org
Subject: [libfabric-users] FI_EP_DGRAM using fi_verbs provider

Hi Libfabric community,

Can anyone point me to an example of how to use DGRAM with the fi_verbs provider?

I have no problem getting DGRAM to work with the fi_udp provider, but when I move the code to verbs, the address resolution fails.

I can see that DGRAM supports/needs FI_ADDR_IB_UD, but I am unable to add such addresses to the AV.

Furthermore, the verbs provider doc refers to variables specific to DGRAM endpoints:

FI_VERBS_DGRAM_USE_NAME_SERVER:  The option that enables/disables OFI Name Server thread. The NS thread is used to resolve IP-addresses to provider specific addresses (default: 1, if “OMPI_COMM_WORLD_RANK” and “PMI_RANK” environment variables aren’t defined)
FI_VERBS_NAME_SERVER_PORT: The port on which Name Server thread listens incoming connections and requests (default: 5678)

This makes it sound like ip address resolution should work, but I cannot find any documentation on this.

Thanks Anders

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20240811/5559f486/attachment-0001.htm>