<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Dear List,</p>
<p><br>
</p>
<p>I have a test that seems to run fine on <span>tcp;ofi_rxm</span> - though this test is two ranks on the same laptop, so it isn't really a very good test - however, I can throw anything at it and it seems to reliably complete.</p>
<p><br>
</p>
<p>On GNI, I get lockups and after much head scratching, I am wondering what the significance of the tx/rx attribute size may be.</p>
<p>On tcp/ofi_rxm the size reports as "<span>size: 65536" and I can have 16 threads each sending up to 128 messages in flight on one thread per endpoint, and a single receive endpoint handling all receives - possibly 16*128 messages with posted receives = 2048.</span></p>
<p><span><br>
</span></p>
<p><span>When I run on GNI, using two nodes, each reports tx/rx attr "<span>size: 500</span>" - and I find that when many messages are in flight, things can lock up because some posted sends are never received. This seems to happen even when I drop down to
16 threads with 8 in flight messages which ought to be 128 at a time - and I would have suspected that a size of 500 (cq size limitation?) would handle this.
</span></p>
<p><br>
</p>
<p>Question 1 - what is the tx/rx attr size really telling me?</p>
<p>Question 2 - if I post more than the allowed receives or sends, should I not receive some kind of error? (I have enabled resource management, so I might expect a retry code when I attempt the send/recv)</p>
<p><br>
</p>
<p>Ideally, I'd like to throttle the number of messages in flight according to what the hardware reports its capabilities - which vars should I use from the fi_info to do this?</p>
<p><br>
</p>
<p>thanks</p>
<p><br>
</p>
<p>JB</p>
<p><br>
</p>
</div>
</body>
</html>