[libfabric-users] fi_cq_open fails on the verbs provider

[Hefty, Sean]

> error: fi_connect failed: -263 'Missing or unavailable completion queue'
> 
> 
> >   if(ret = fi_connect(socket->ep, socket->fi->dest_addr, NULL, 0)) {
> >     ERROR("fi_connect failed: %d '%s'", ret, strerror(-ret));
> >   }

The connect call enables the endpoint.  That requires that the endpoint be bound to the send and/or receive CQs.

Also, please see:

https://github.com/ofiwg/libfabric/pull/1780
https://github.com/ofiwg/libfabric/issues/1645

regarding trying to use native wait objects with epoll.  The current API is basically broken for supporting this usage model.

A proposed fix is available, but has not been merged yet, and will not be available until the 1.3 release next month.

- Sean