[libfabric-users] Can only use one NIC port in libfabric 1.6.1

[Jörn Schumacher]

Dear libfabric developers,

I recently updated to libfabric 1.6.1 (from 1.4). It looks like in this 
release we can only use on port of our NIC (Mellanox ConnectX-5 with RoCE).

On the receiving side we listen for a RC. We monitor the event queue 
with a file descriptor + epoll. On one port of the NIC this works fine, 
but if the request comes in on the second port (on a different IP 
subnet) this fails: we get an epoll notification, but then the 
subsequent fi_eq_sread(...) call yields FI_EAGAIN.

I open a single domain. This worked fine in the earlier libfabric. 
Reading the documentation a bit I understand that a domain is tied to a 
port. Does this mean I need to open multiple domains?


Thanks and best regards,
Jörn