<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=EN-US link=blue vlink="#954F72"><div class=WordSection1><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Hello everyone,</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I could use some help with this issue. </p><p class=MsoNormal>That’s my little project I’m working on: <a href="https://github.com/germanafro/Libfabric_Benchmark">https://github.com/germanafro/Libfabric_Benchmark</a></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>A Benchmark one day hoping to achieve Bandwidths of up to 1TB/s though apparently I’m struggling with the basic setup still. </p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I started testing and implementation from home with the sockets provider. So far everything seemed to work just fine, once I moved to my universities testcluster the sockets provider still does what it’s supposed to do but the transition to the verbs provider runs me into one bump after another :/ </p><p class=MsoNormal>The testcluster itself consist of ~40 linux nodes connected through ethernet and Infiniband switches.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>With the most recent issue of: <b>fi_cq_sread: Resource temporarily unavailable</b></p><p class=MsoNormal style='text-align:justify;text-justify:inter-ideograph'><b>fi_cq_open(…) </b>and <b>fi_ep_bind(…) </b>seem to be fine.</p><p class=MsoNormal style='text-align:justify;text-justify:inter-ideograph'><o:p> </o:p></p><p class=MsoNormal>Could it be my config or is this actually a temporary network issue? I tested several clusters.</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>//Completion queue config</p><p class=MsoNormal> cq_attr.size = 0;</p><p class=MsoNormal> cq_attr.flags = 0;</p><p class=MsoNormal> cq_attr.format = FI_CQ_FORMAT_MSG;</p><p class=MsoNormal> cq_attr.wait_obj = FI_WAIT_UNSPEC;</p><p class=MsoNormal> cq_attr.signaling_vector = 0;</p><p class=MsoNormal> cq_attr.wait_cond = FI_CQ_COND_NONE;</p><p class=MsoNormal> cq_attr.wait_set = NULL;</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I appreciate the help and time,</p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Andreas</p><p class=MsoNormal><o:p> </o:p></p></div></body></html>