<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<script type="text/javascript">window["_gaUserPrefs"] = { ioo : function() { return true; } }</script><style type="text/css" id="owaParaStyle">
<!--
p
{margin-top:0;
margin-bottom:0}
p
{margin-top:0;
margin-bottom:0}
-->
P {margin-top:0;margin-bottom:0;}</style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">
<div>I believe I understand the problem now.</div>
<div><br>
</div>
<div>My original implementation was correct, but when the first message arrives, the receiving node does not have it's address in the address vector yet and so it reports FI_ADDR_NOTAVAIL.
<br>
</div>
<div>I'm converting our bootstrap routine that used PMI on cray to work with sockets on other machines. I shall send an extra message first with only the socket information needed to the root node and it can then insert the correct address into the av, then
things can resume as before (hopefully).</div>
<div><br>
</div>
<div>JB<br>
</div>
<div><br>
</div>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div id="divRpF832102" style="direction: ltr;"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Libfabric-users [libfabric-users-bounces@lists.openfabrics.org] on behalf of Biddiscombe, John A. [biddisco@cscs.ch]<br>
<b>Sent:</b> 10 March 2019 11:28<br>
<b>To:</b> libfabric-users@lists.openfabrics.org<br>
<b>Subject:</b> Re: [libfabric-users] trouble by FI_SOURCE use<br>
</font><br>
</div>
<div></div>
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">
<div>I think I have the concept of source/dest the wrong way around and I should be using the dest field, since I want to receive on that port number.
<br>
</div>
<div><br>
</div>
<div>Am I correct in thinking that when creating the endpoint, the source field is me if I'm sending, but the dest field if I'm receiving on a particular port?</div>
<div><br>
</div>
<div>thanks</div>
<div><br>
</div>
<div>JB<br>
</div>
<div style="font-family:Times New Roman; color:#000000; font-size:16px">
<hr tabindex="-1">
<div id="divRpF224677" style="direction:ltr"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Libfabric-users [libfabric-users-bounces@lists.openfabrics.org] on behalf of Biddiscombe, John A. [biddisco@cscs.ch]<br>
<b>Sent:</b> 10 March 2019 10:21<br>
<b>To:</b> libfabric-users@lists.openfabrics.org<br>
<b>Subject:</b> [libfabric-users] trouble by FI_SOURCE use<br>
</font><br>
</div>
<div></div>
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">
<div>Hello list</div>
<div><br>
</div>
<div>I need to use a particular port number when setting up a connection, so I have</div>
<div><br>
</div>
<div>fabric_hints_->src_addr = socket_data;<br>
fabric_hints_->addr_format = FI_SOCKADDR_IN;<br>
fabric_hints_->src_addrlen = sizeof(struct sockaddr_in);<br>
//<br>
fabric_hints_->caps = FI_MSG | FI_RMA | FI_SOURCE |<br>
FI_WRITE | FI_READ | FI_REMOTE_READ | FI_REMOTE_WRITE | FI_RMA_EVENT;<br>
</div>
<div><br>
</div>
<div>and all is well, but when a message comes in - <br>
</div>
<div>if (src_addr == FI_ADDR_NOTAVAIL)<br>
{<br>
LOG_DEBUG_MSG("Source address not available...\n");<br>
std::terminate();<br>
}<br>
</div>
<div><br>
</div>
<div>my check for the source fails. This is documented - however the documentation/examples appear ambiguous</div>
<div>
<dl><dt><em>"src_addr - source address</em></dt><dd>If specified, indicates the source address. This field will be ignored in hints if FI_SOURCE flag is set."</dd><dt><br>
</dt><dt>But the examples/tutorials use src_addr and still set FI_SOURCE, so I tried it too and the endpoint is created on the correct port number. I'm happy.<br>
</dt></dl>
<div>however, <br>
</div>
<div>"<em>FI_SOURCE</em>
<dl><dd>Requests that the endpoint return source addressing data as part of its completion data."</dd></dl>
<div>So if I need the endpoint on a certain port number and I set src_addr, but now the FI_SOURCE flag is dropped, but I also want to know where messages are coming from - how do I get the source information with the message?</div>
<div><br>
</div>
<div>Apologies if this question appears trivial, I've not looked at the code for a long time and forgotten much of what I knew about libfabric.</div>
<div><br>
</div>
<div>yours</div>
<div><br>
</div>
<div>JB</div>
<div><br>
</div>
</div>
</div>
<div><br>
<div style="font-family:Tahoma; font-size:13px"><font size="2" face="Courier New">--
<br>
Dr. John Biddiscombe, email:biddisco @.at.@ cscs.ch<br>
http://www.cscs.ch/<br>
CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07<br>
Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82<br>
</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>