<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr" style="text-align:left; direction:ltr;">
<div>Hi Greg,</div>
<div><br>
</div>
<div>The man page is correct in that the RDM endpoint does not require memory registration. The EFA device does require registrations but the Libfabric provider will do the registration on behalf of the user or copy into pre-registered bounce buffers depending
on the protocol used.</div>
<div><br>
</div>
<div>There are examples of Libfabric users that require this behavior, for example the Open MPI OFI MTL does not do any memory registrations. I'd cut an issue on GitHub for the error and we can take a look!</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Robert</div>
<div><br>
</div>
<div>On Mon, 2020-07-27 at 22:53 +0000, Titus, Greg wrote:</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div class="WordSection1">
<table class="MsoTableGrid" border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody>
<tr style="height:15.25pt">
<td width="711" valign="top" style="width:842.35pt;border:solid #ED7D31 1.5pt;padding:0in 5.4pt 0in 5.4pt;height:15.25pt">
<p><strong><span style="background:#FFFF99">CAUTION</span></strong><span style="background:#FFFF99">: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
</div>
<br>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Greetings --</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I'm trying to use the efa provider on an AWS instance that has the Elastic Fabric Adaptor high-performance network interface. In the fi_efa(7) man page for libfabric version 1.9.0amzn1.1 installed on the system it says:</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
The RDM endpoint does not require memory registration<br>
</div>
</blockquote>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I was a little in disbelief when I read that, and thus was not surprised when I tried simply not making any memory registration calls and sure enough, got internal errors in libfabric. So, w<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">hat
does that statement in the man page mean, exactly? <span> I</span></span>s there maybe a magic remote key value that means "I didn't register memory."? I'<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">m
just using 0UL, in the absence of any other guidance.</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;"><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">thanks,</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">greg</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;"><br>
</span></div>
</div>
<pre>_______________________________________________</pre>
<pre>Libfabric-users mailing list</pre>
<pre><a href="mailto:Libfabric-users@lists.openfabrics.org">Libfabric-users@lists.openfabrics.org</a></pre>
<pre><a href="https://lists.openfabrics.org/mailman/listinfo/libfabric-users">https://lists.openfabrics.org/mailman/listinfo/libfabric-users</a></pre>
<pre><br></pre>
</blockquote>
</body>
</html>