<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:10.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#467886;
text-decoration:underline;}
span.EmailStyle200
{mso-style-type:personal-reply;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">I assumed PSM3 underneath uses rdma-core for RoCE protocol.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Slide 8 from <a href="https://www.openfabrics.org/wp-content/uploads/2021-workshop-presentations/405_Rimmer_PSM3-Architecture.pdf">
https://www.openfabrics.org/wp-content/uploads/2021-workshop-presentations/405_Rimmer_PSM3-Architecture.pdf</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">So, PSM3 has its own implementation of RoCE protocol?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Xiong, Jianxin <jianxin.xiong@intel.com><br>
<b>Date: </b>Wednesday, August 21, 2024 at 10:34</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:12.0pt;color:black">AM<br>
<b>To: </b>Niyaz Murshed <Niyaz.Murshed@arm.com>, libfabric-users@lists.openfabrics.org <libfabric-users@lists.openfabrics.org><br>
<b>Subject: </b>RE: PD/MR sharing by multiple NIC<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">As I said, you can’t do that with the verbs provider.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Niyaz Murshed <Niyaz.Murshed@arm.com>
<br>
<b>Sent:</b> Wednesday, August 21, 2024 8:28 AM<br>
<b>To:</b> Xiong, Jianxin <jianxin.xiong@intel.com>; libfabric-users@lists.openfabrics.org<br>
<b>Subject:</b> Re: PD/MR sharing by multiple NIC</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Thank you Jianxin.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">ibv_alloc_pd takes in ibv_context which comes opening a particular device.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">How can we have same PD linked to multiple device ?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Do we have different PDs but same MR?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div id="mail-editor-reference-message-container">
<div>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Xiong, Jianxin <<a href="mailto:jianxin.xiong@intel.com">jianxin.xiong@intel.com</a>><br>
<b>Date: </b>Wednesday, August 21, 2024 at 10:16</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:black"> </span><span style="font-size:12.0pt;color:black">AM<br>
<b>To: </b>Niyaz Murshed <<a href="mailto:Niyaz.Murshed@arm.com">Niyaz.Murshed@arm.com</a>>,
<a href="mailto:libfabric-users@lists.openfabrics.org">libfabric-users@lists.openfabrics.org</a> <<a href="mailto:libfabric-users@lists.openfabrics.org">libfabric-users@lists.openfabrics.org</a>><br>
<b>Subject: </b>RE: PD/MR sharing by multiple NIC</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">It depends on the provider. Since MR is domain level object, in order to share the MR, the endpoints must share the same domain. Some providers (e.g. verbs) map domain to a specific nic and thus can’t share
MR among nics. Other providers (e.g. psm3) have domain as pure software entity and can have multiple NICs under the same domain.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Jianxin</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Libfabric-users <<a href="mailto:libfabric-users-bounces@lists.openfabrics.org">libfabric-users-bounces@lists.openfabrics.org</a>>
<b>On Behalf Of </b>Niyaz Murshed<br>
<b>Sent:</b> Wednesday, August 21, 2024 8:00 AM<br>
<b>To:</b> <a href="mailto:libfabric-users@lists.openfabrics.org">libfabric-users@lists.openfabrics.org</a><br>
<b>Subject:</b> [libfabric-users] PD/MR sharing by multiple NIC</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Hello, </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Is it possible to multiple NICs share the same PD/MR?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">i.e., can I register a memory region that can be used by multiple NICs?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Regards,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Niyaz</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:12.0pt">IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose
the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt">IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose
the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose,
or store or copy the information in any medium. Thank you.
</body>
</html>