<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:202443533;
mso-list-type:hybrid;
mso-list-template-ids:988058062 403243025 403243033 403243035 403243023 403243033 403243035 403243023 403243033 403243035;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> ofw-bounces@lists.openfabrics.org [mailto:ofw-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>Deepak Rawal<br>
<b>Sent:</b> Friday, June 14, 2013 6:24 PM<br>
<b>To:</b> ofw@lists.openfabrics.org<br>
<b>Subject:</b> [ofw] Current recommended APIs for kernel and user<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">For user mode API, there are several options as indicated. However, there is NO kernel API available. It would be very useful if someone could comment on what is possible
/ available for Windows 2012 as well as other (newer and older) version of windows.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="color:#1F497D">As stated before, MS embraces NetworkDirect API for user & kernel mode; talk to MS about NDKPI consumer interface as it is not publically released.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="color:#1F497D">winOFED / IBAL has a kernel API utilized by the winOFED and Mellanox IPoIB drivers.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="color:#1F497D">Since the Mellanox stack and winOFED are derived from the same code base, you should talk with Mellanox about their plans for supporting kernel-mode IBAL and user-mode IBAL along with user-mode ibverbs
support (Linux verbs compat).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">It would be greatly appreciated if some guidance on kernel API, even as possibility, is indicated by the team (e.g. if any members use kernel RDMA for their solution
and plan to continue to use it, what plan they recommend)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><span style="mso-list:Ignore">1)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">MS has published preliminary info on NDKPI interface (part of NDIS extension) which can be used by providers and consumers, but consumer interface is “closed”.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><span style="mso-list:Ignore">2)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">There is some alternate information available on MS website that refers to kRDMA, but it is not “found” in WDK.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">Deepak Rawal<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">I don't know if anyone responded to this, but in case not...<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> We are working on a project that involves both kernel and user mode code<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> communicating over Mellanox adapters (RoCE mode) on Windows Server 2012. We do<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> need to interoperate with Linux. This is for a commercial datacenter product.<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i>
<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i>
<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i>
<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> I see there are multiple APIs that can be used, and am wondering what the OFW<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> team is currently recommending for new code?<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i>
<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i>
<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i>
<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> It seems like the options include: NDv1, NDv2, IBAL, WinVerbs, libibverbs.<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> Compatibility with preexisting Linux code is useful, but having maximum<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> performance likely is more important. It seems like NDv1/NDv2 is officially<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> supported by Microsoft in user mode, and there currently is no officially<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">><i> supported Microsoft kernel RDMA API.<o:p></o:p></i></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">ND is the official API defined by MS. I believe that there's a kernel ND API in Server 2012, but someone from MS will need to confirm.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">RoCE support comes directly from Mellanox, so you would need to ask them what APIs they will support.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">In addition to the APIs you list, there is a userspace version of the Linux libibverbs available on windows as part of WinOFED.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE" style="font-size:10.0pt;font-family:"Courier New"">- Sean<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-IE"><o:p> </o:p></span></p>
</div>
</div>
</body>
</html>