<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<script type="text/javascript">window["_gaUserPrefs"] = { ioo : function() { return true; } }</script><style type="text/css" id="owaParaStyle">
<!--
p
{margin-top:0;
margin-bottom:0}
p
{margin-top:0;
margin-bottom:0}
p
{margin-top:0;
margin-bottom:0}
-->
P {margin-top:0;margin-bottom:0;}</style>
</head>
<body fpstyle="1" ocsi="0">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">
<div>Question:</div>
<div>Is it possible to setup a connection using only a connectionless endpoint at both ends?</div>
<div><br>
</div>
<div>Using a single node (for testing) I'm doing this<br>
</div>
<div>process 0 : create an rdm connection on one end using src = 127.0.0.1:11111 and dst = null, and all appears fine.</div>
<div>process 1: create an connection using dst = 127.0.0.1:11111 and src=null, all appears fine. After creating the endpoint, I can see that this end has been assigned a port number of 22222.</div>
<div>process 1 : creates an address vector with its own address in it and the address of process 0 : 127.0.0.1:11111 in it. sends it's own address 127.0.0.1:22222 to process 0 which receives it correctly (with the ADDR NOT AVAIL flag set)<br>
</div>
<div>Process 0 now adds the address 127:0.0.1:22222 to its address vector and sends a 'hello' message to process 1:</div>
<div>process 1 does not receive this message. But process 0 does receive it.</div>
<div><br>
</div>
<div>Why is the message going from process 0 back to process 0 and not to process 1?</div>
<div><br>
</div>
<div>Thanks</div>
<div><br>
</div>
<div>JB<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<hr tabindex="-1">
<div id="divRpF270587" style="direction: ltr;"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Libfabric-users [libfabric-users-bounces@lists.openfabrics.org] on behalf of Biddiscombe, John A. [biddisco@cscs.ch]<br>
<b>Sent:</b> 10 March 2019 14:53<br>
<b>To:</b> libfabric-users@lists.openfabrics.org<br>
<b>Subject:</b> Re: [libfabric-users] trouble by FI_SOURCE use<br>
</font><br>
</div>
<div></div>
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">
<div>I believe I understand the problem now.</div>
<div><br>
</div>
<div>My original implementation was correct, but when the first message arrives, the receiving node does not have it's address in the address vector yet and so it reports FI_ADDR_NOTAVAIL.
<br>
</div>
<div>I'm converting our bootstrap routine that used PMI on cray to work with sockets on other machines. I shall send an extra message first with only the socket information needed to the root node and it can then insert the correct address into the av, then
things can resume as before (hopefully).</div>
<div><br>
</div>
<div>JB<br>
</div>
<div><br>
</div>
<div style="font-family:Times New Roman; color:#000000; font-size:16px">
<hr tabindex="-1">
<div id="divRpF832102" style="direction:ltr"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Libfabric-users [libfabric-users-bounces@lists.openfabrics.org] on behalf of Biddiscombe, John A. [biddisco@cscs.ch]<br>
<b>Sent:</b> 10 March 2019 11:28<br>
<b>To:</b> libfabric-users@lists.openfabrics.org<br>
<b>Subject:</b> Re: [libfabric-users] trouble by FI_SOURCE use<br>
</font><br>
</div>
<div></div>
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">
<div>I think I have the concept of source/dest the wrong way around and I should be using the dest field, since I want to receive on that port number.
<br>
</div>
<div><br>
</div>
<div>Am I correct in thinking that when creating the endpoint, the source field is me if I'm sending, but the dest field if I'm receiving on a particular port?</div>
<div><br>
</div>
<div>thanks</div>
<div><br>
</div>
<div>JB<br>
</div>
<div style="font-family:Times New Roman; color:#000000; font-size:16px">
<hr tabindex="-1">
<div id="divRpF224677" style="direction:ltr"><font size="2" face="Tahoma" color="#000000"><b>From:</b> Libfabric-users [libfabric-users-bounces@lists.openfabrics.org] on behalf of Biddiscombe, John A. [biddisco@cscs.ch]<br>
<b>Sent:</b> 10 March 2019 10:21<br>
<b>To:</b> libfabric-users@lists.openfabrics.org<br>
<b>Subject:</b> [libfabric-users] trouble by FI_SOURCE use<br>
</font><br>
</div>
<div></div>
<div>
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">
<div>Hello list</div>
<div><br>
</div>
<div>I need to use a particular port number when setting up a connection, so I have</div>
<div><br>
</div>
<div>fabric_hints_->src_addr = socket_data;<br>
fabric_hints_->addr_format = FI_SOCKADDR_IN;<br>
fabric_hints_->src_addrlen = sizeof(struct sockaddr_in);<br>
//<br>
fabric_hints_->caps = FI_MSG | FI_RMA | FI_SOURCE |<br>
FI_WRITE | FI_READ | FI_REMOTE_READ | FI_REMOTE_WRITE | FI_RMA_EVENT;<br>
</div>
<div><br>
</div>
<div>and all is well, but when a message comes in - <br>
</div>
<div>if (src_addr == FI_ADDR_NOTAVAIL)<br>
{<br>
LOG_DEBUG_MSG("Source address not available...\n");<br>
std::terminate();<br>
}<br>
</div>
<div><br>
</div>
<div>my check for the source fails. This is documented - however the documentation/examples appear ambiguous</div>
<div>
<dl><dt><em>"src_addr - source address</em></dt><dd>If specified, indicates the source address. This field will be ignored in hints if FI_SOURCE flag is set."</dd><dt><br>
</dt><dt>But the examples/tutorials use src_addr and still set FI_SOURCE, so I tried it too and the endpoint is created on the correct port number. I'm happy.<br>
</dt></dl>
<div>however, <br>
</div>
<div>"<em>FI_SOURCE</em>
<dl><dd>Requests that the endpoint return source addressing data as part of its completion data."</dd></dl>
<div>So if I need the endpoint on a certain port number and I set src_addr, but now the FI_SOURCE flag is dropped, but I also want to know where messages are coming from - how do I get the source information with the message?</div>
<div><br>
</div>
<div>Apologies if this question appears trivial, I've not looked at the code for a long time and forgotten much of what I knew about libfabric.</div>
<div><br>
</div>
<div>yours</div>
<div><br>
</div>
<div>JB</div>
<div><br>
</div>
</div>
</div>
<div><br>
<div style="font-family:Tahoma; font-size:13px"><font size="2" face="Courier New">--
<br>
Dr. John Biddiscombe, email:biddisco @.at.@ cscs.ch<br>
http://www.cscs.ch/<br>
CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07<br>
Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82<br>
</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>