[libfabric-users] fi_cq_open fails on the verbs provider

[Ilango, Arun]

Hi Jörn,

Currently, verbs provider requires an endpoint to be bound to a CQ for both transmit and receive operations. This is a bug in the provider and I have pushed a fix here: https://github.com/ofiwg/libfabric/pull/1782. Please update your libfabric once the fix is merged and your code should work. 

Thanks,
Arun.

-----Original Message-----
From: Hefty, Sean 
Sent: Wednesday, February 24, 2016 2:20 PM
To: Jörn Schumacher <joern.schumacher at cern.ch>; Ilango, Arun <arun.ilango at intel.com>; libfabric-users at lists.openfabrics.org
Subject: RE: [libfabric-users] fi_cq_open fails on the verbs provider

> error: fi_connect failed: -263 'Missing or unavailable completion queue'
> 
> 
> >   if(ret = fi_connect(socket->ep, socket->fi->dest_addr, NULL, 0)) {
> >     ERROR("fi_connect failed: %d '%s'", ret, strerror(-ret));
> >   }

The connect call enables the endpoint.  That requires that the endpoint be bound to the send and/or receive CQs.

Also, please see:

https://github.com/ofiwg/libfabric/pull/1780
https://github.com/ofiwg/libfabric/issues/1645

regarding trying to use native wait objects with epoll.  The current API is basically broken for supporting this usage model.

A proposed fix is available, but has not been merged yet, and will not be available until the 1.3 release next month.

- Sean