[libfabric-users] verbs provider completion queue: "fi_cq_sread: Resource temporarily unavailable"

[Andreas]

Hello everyone,

I could use some help with this issue. 
That’s my little project I’m working on: https://github.com/germanafro/Libfabric_Benchmark

A Benchmark one day hoping to achieve Bandwidths of up to 1TB/s though apparently I’m struggling with the basic setup still. 

I started testing and implementation from home with the sockets provider. So far everything seemed to work just fine, once I moved to my universities testcluster the sockets provider still does what it’s supposed to do but the transition to the verbs provider runs me into one bump after another :/ 
The testcluster itself consist of ~40 linux nodes connected through ethernet and Infiniband switches.

With the most recent issue of:  fi_cq_sread: Resource temporarily unavailable
fi_cq_open(…) and fi_ep_bind(…) seem to be fine.

Could it be my config or is this actually a temporary network issue? I tested several clusters.

//Completion queue config
	cq_attr.size = 0;
	cq_attr.flags = 0;
	cq_attr.format = FI_CQ_FORMAT_MSG;
	cq_attr.wait_obj = FI_WAIT_UNSPEC;
	cq_attr.signaling_vector = 0;
	cq_attr.wait_cond = FI_CQ_COND_NONE;
	cq_attr.wait_set = NULL;


I appreciate the help and time,

Andreas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/libfabric-users/attachments/20181208/67e6f06a/attachment.html>