<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:Calibri;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:Calibri;
mso-fareast-language:EN-US;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:14.0pt">I’ve converted my code to use RDM endpoints instead of MSG ones and have a conceptual problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">Node A knows the address of Node B at startup due to some out of band exchange.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">Node A adds the address of B to an address vector and gets an fi_addr_t which it then uses to send a message to B.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">The message contains some rdma info to tell B “grab this chunk of data from me at this address”<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">With a connection oriented mode, Node B has a connection to A and uses the ‘A endpoint’ it created after FI_CONNECT to do an fi_read from A.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">With connectionless, Node B has no real idea node A exists and receives the message saying “grab this data” (from where?).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">What is the recommended way for B to get the data from A. It has not added the address of A to its address vector and has no fi_addr_t for A and when it receives a message from A – so how does it know who
A is?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">Does A have to send (on first connection/message) a warning message to B so that B can add A it to its AV first? Is this how it is normally done? (this would mimic the connect request/connected model used
for MSG endpoint).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">Or … should every node add every other node to its AV at startup so that when B gets a pull request from A it at least has an fi_addr_t of A and only needs to know which node to pull from. (If I have thousands
of nodes communicating, but only using a nearest neighbour pattern, it seems a waste to add N^2 addresses to AV at startup)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">When I receive a message on an RDM endpoint, the same endpoint handles all peers, so now I don’t know how to get the address of the message origin again – do I have to send the address with every rdma request
so that the receiver know where to read from?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">PS. Extra question. If Node A registers node C in its AV and node B registers node C in its AV, will A dna B have the same fi_addr_t for C? (Can they be shared? – imlying that I could send the fi_addr_t as
a handle with my rdma read requests so that B would know what address to use when getting data)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">Sorry for so many questions, and thanks in advance for any advice<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt">JB<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:"Times New Roman";mso-fareast-language:EN-GB">--</span><span style="font-family:"Times New Roman";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:"Times New Roman";mso-fareast-language:EN-GB">John Biddiscombe, email:biddisco @.at.@ cscs.ch</span><span style="font-family:"Times New Roman";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:"Times New Roman";mso-fareast-language:EN-GB"><a href="http://www.cscs.ch/"><span style="color:#0563C1">http://www.cscs.ch/</span></a></span><span style="font-family:"Times New Roman";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-family:"Times New Roman";mso-fareast-language:EN-GB">CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07</span><span style="font-family:"Times New Roman";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="IT-CH" style="font-family:"Times New Roman";mso-fareast-language:EN-GB">Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82</span><span style="font-family:"Times New Roman";mso-fareast-language:EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>