<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:"Segoe UI"}
@font-face
{font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
a:link, span.MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
span.BalloonTextChar
{font-family:"Tahoma","sans-serif"}
p.msochpdefault, li.msochpdefault, div.msochpdefault
{margin-right:0in;
margin-left:0in;
font-size:10.0pt;
font-family:"Times New Roman","serif"}
p.Sprechblasentext, li.Sprechblasentext, div.Sprechblasentext
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif"}
span.SprechblasentextZchn
{font-family:"Segoe UI","sans-serif"}
span.emailstyle17
{font-family:"Calibri","sans-serif";
color:windowtext}
span.emailstyle18
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.balloontextchar0
{font-family:"Tahoma","sans-serif"}
span.emailstyle22
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle26
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle27
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle28
{font-family:"Calibri","sans-serif";
color:#1F497D}
.MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.WordSection1
{}
-->
</style>
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div>
<div style="font-size:11pt; font-family:Calibri,sans-serif">Hi Fab,<br>
<br>
thanks. I already got access to the IDN2Provider on Friday - the headers are also included in Mellanox' distribution just as Tzachi said. The problem on my side was that I was looking for some import libraries. I finally found some NDv1 documentation on MSDN
explaining how to get the root object via the Winsock provider catalogue. However, there does not seem to be any NDv2 documentation yet, but the differences seem to be subtle.<br>
<br>
As I have only limited time, I will probably not be able to rewrite all of my code at once. I think that my NDv2 code will be able to talk to librdma peers as long as I use only basic functionality like remote read/write or inline send/receive. Is that correct?<br>
<br>
Best regards,<br>
Christoph <br>
<br>
Von meinem Windows Phone gesendet</div>
</div>
<div dir="ltr">
<hr>
<span style="font-size:11pt; font-family:Calibri,sans-serif; font-weight:bold">Von:
</span><span style="font-size:11pt; font-family:Calibri,sans-serif"><a href="mailto:ftillier@microsoft.com">Fab Tillier</a></span><br>
<span style="font-size:11pt; font-family:Calibri,sans-serif; font-weight:bold">Gesendet:
</span><span style="font-size:11pt; font-family:Calibri,sans-serif">18.08.2013 07:38</span><br>
<span style="font-size:11pt; font-family:Calibri,sans-serif; font-weight:bold">An:
</span><span style="font-size:11pt; font-family:Calibri,sans-serif"><a href="mailto:christoph.mueller@visus.uni-stuttgart.de">Christoph Müller</a></span><br>
<span style="font-size:11pt; font-family:Calibri,sans-serif; font-weight:bold">Cc:
</span><span style="font-size:11pt; font-family:Calibri,sans-serif"><a href="mailto:ofw@lists.openfabrics.org">'ofw@lists.openfabrics.org'</a></span><br>
<span style="font-size:11pt; font-family:Calibri,sans-serif; font-weight:bold">Betreff:
</span><span style="font-size:11pt; font-family:Calibri,sans-serif">RE: Use of I/O completion port with Winverbs</span><br>
<br>
</div>
<div>
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Christoph,</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">You can get the ND header files (included in the NetworkDirect DDK zip file) here:</span></p>
<p class="MsoNormal"><span style="color:#1F497D"><a href="http://www.microsoft.com/en-us/download/details.aspx?id=36043">http://www.microsoft.com/en-us/download/details.aspx?id=36043</a></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">There is no dependency on the HPC Pack - you just need drivers that support ND (ideally NDv2, which the Mellanox 4.40 release does).</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">Using ND should give you the greatest hardware independence too, though by design ND exposes the lowest common denominator functionality of iWARP and InfiniBand, so some functionality is not available (e.g. RDMA
atomics).</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">Don’t hesitate to ask if you have questions.</span></p>
<p class="MsoNormal"><span style="color:#1F497D">-Fab</span></p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="color:#1F497D"> </span></a></p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> ofw-bounces@lists.openfabrics.org [mailto:ofw-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>Christoph Müller<br>
<b>Sent:</b> Thursday, August 15, 2013 6:04 AM<br>
<b>To:</b> Tzachi Dar; ofw@lists.openfabrics.org<br>
<b>Subject:</b> Re: [ofw] Use of I/O completion port with Winverbs</p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span lang="EN-GB" style="color:#1F497D">Thank you. I was not aware that the SDK including NDv2 is distributed with Mellanox’ drivers. In this case, I will give it a try.</span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:#1F497D">Thanks again,</span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:#1F497D">Christoph</span></p>
<p class="MsoNormal"><span lang="EN-GB" style="color:#1F497D"> </span></p>
<div style="border:none; border-left:solid blue 1.5pt; padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span lang="EN-GB">Von:</span></b><span lang="EN-GB"> Tzachi Dar [<a href="mailto:tzachid@mellanox.com">mailto:tzachid@mellanox.com</a>]
<br>
<b>Gesendet:</b> Donnerstag, 15. August 2013 14:43<br>
<b>An:</b> Ch</span><span lang="DE">ristoph Müller; <a href="mailto:ofw@lists.openfabrics.org">
ofw@lists.openfabrics.org</a><br>
<b>Betreff:</b> RE: Use of I/O completion port with Winverbs</span></p>
</div>
</div>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">You only need to download Mellanox drivers 4.40 for windows, and you will have nd support.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"><a href="http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers">http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers</a></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks</span></p>
<p class="MsoNormal"><span style="color:#1F497D">Tzachi</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Christoph Müller [<a href="mailto:christoph.mueller@visus.uni-stuttgart.de">mailto:christoph.mueller@visus.uni-stuttgart.de</a>]
<br>
<b>Sent:</b> Thursday, August 15, 2013 3:32 PM<br>
<b>To:</b> Tzachi Dar; <a href="mailto:ofw@lists.openfabrics.org">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> AW: Use of I/O completion port with Winverbs</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal">ndinstall is part of the HPC pack distribution? My main problem with ND is to find out how to get it...<br>
<br>
Best regards,<br>
Christoph<br>
<br>
Von meinem Windows Phone gesendet</p>
</div>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt; font-family:"Times New Roman","serif"">
<hr size="2" width="100%" align="center">
</span></div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b>Von: </b><a href="mailto:tzachid@mellanox.com">Tzachi Dar</a><span style="font-size:12.0pt; font-family:"Times New Roman","serif""><br>
</span><b>Gesendet: </b><span style="font-family:"Arial","sans-serif""></span>15.<span style="font-family:"Arial","sans-serif""></span>08.<span style="font-family:"Arial","sans-serif""></span>2013 14:28<span style="font-size:12.0pt; font-family:"Times New Roman","serif""><br>
</span><b>An: </b><a href="mailto:christoph.mueller@visus.uni-stuttgart.de">Christoph Müller</a>;
<a href="mailto:ofw@lists.openfabrics.org">ofw@lists.openfabrics.org</a><span style="font-size:12.0pt; font-family:"Times New Roman","serif""><br>
</span><b>Betreff: </b>RE: Use of I/O completion port with Winverbs<span style="font-size:12.0pt; font-family:"Times New Roman","serif""></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Please note that the windows APIs are changing from time to time and ND is probably the one that will stay longer since it is supported by MS.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">I don’t think that you need to install the hpc pack for it. You simply need to run ndinstall and then you can start running nd programs.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks</span></p>
<p class="MsoNormal"><span style="color:#1F497D">Tzachi</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> Christoph Müller [<a href="mailto:christoph.mueller@visus.uni-stuttgart.de">mailto:christoph.mueller@visus.uni-stuttgart.de</a>]
<br>
<b>Sent:</b> Thursday, August 15, 2013 3:21 PM<br>
<b>To:</b> Tzachi Dar; <a href="mailto:ofw@lists.openfabrics.org">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> AW: Use of I/O completion port with Winverbs</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<div>
<div>
<p class="MsoNormal">Actually, I am in the process of deciding which API to use (although I have existing librdma code). NetworkDirect indeed looks promising, but as I understand, I need HPC Pack 2012 for that which I cannot install on our cluster in the short
term.<br>
<br>
Best regards,<br>
Christoph<br>
<br>
<br>
Von meinem Windows Phone gesendet</p>
</div>
</div>
<div>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:12.0pt; font-family:"Times New Roman","serif"">
<hr size="2" width="100%" align="center">
</span></div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b>Von: </b><a href="mailto:tzachid@mellanox.com">Tzachi Dar</a><span style="font-size:12.0pt; font-family:"Times New Roman","serif""><br>
</span><b>Gesendet: </b><span style="font-family:"Arial","sans-serif""></span>15.<span style="font-family:"Arial","sans-serif""></span>08.<span style="font-family:"Arial","sans-serif""></span>2013 14:15<span style="font-size:12.0pt; font-family:"Times New Roman","serif""><br>
</span><b>An: </b><a href="mailto:christoph.mueller@visus.uni-stuttgart.de">Christoph Müller</a>;
<a href="mailto:ofw@lists.openfabrics.org">ofw@lists.openfabrics.org</a><span style="font-size:12.0pt; font-family:"Times New Roman","serif""><br>
</span><b>Betreff: </b>RE: Use of I/O completion port with Winverbs</p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">can you write your application with Network Direct?</span></p>
<p class="MsoNormal"><span style="color:#1F497D">If so, there are overlapped objects there that you should be able to connect to the completion ports.</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks</span></p>
<p class="MsoNormal"><span style="color:#1F497D">Tzachi</span></p>
<p class="MsoNormal"><span style="color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">
<a href="mailto:ofw-bounces@lists.openfabrics.org">ofw-bounces@lists.openfabrics.org</a> [<a href="mailto:ofw-bounces@lists.openfabrics.org">mailto:ofw-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>Christoph Müller<br>
<b>Sent:</b> Thursday, August 15, 2013 3:10 PM<br>
<b>To:</b> <a href="mailto:ofw@lists.openfabrics.org">ofw@lists.openfabrics.org</a><br>
<b>Subject:</b> [ofw] Use of I/O completion port with Winverbs</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span lang="DE">Hi all,</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">I was wondering whether it is possible to use CreateIoCompletionPort etc. with IWVConnectEndpoints. I can see that the implementation inherits a handle from CWVBase, but I do not see how I could use this information for
creating an IOCP and, of course, I do not know whether it would work. Furthermore, I would prefer continuing to use the RDMA library, but I see absolutely no way to create an IOCP here. So me questions are:</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Is it possible to use IOCPs with winverbs?</span></p>
<p class="MsoNormal"><span lang="EN-GB">Is it possible to use IOCPs with the RDMA library?</span></p>
<p class="MsoNormal"><span lang="EN-GB">Is there sample code for either of it?</span></p>
<p class="MsoNormal"><span lang="EN-GB"> </span></p>
<p class="MsoNormal"><span lang="EN-GB">Thanks in advance,</span></p>
<p class="MsoNormal"><span lang="EN-GB">Christoph</span></p>
<p class="MsoNormal"><span lang="DE" style="font-size:10.5pt">--</span></p>
<p class="MsoNormal"><span lang="DE" style="font-size:10.5pt">Visualisierungsinstitut der Universität Stuttgart</span></p>
<p class="MsoNormal"><span lang="DE" style="font-size:10.5pt">Allmandring 19, 70569 Stuttgart</span></p>
<p class="MsoNormal"><span lang="DE" style="font-size:10.5pt">E-Mail: <a href="mailto:christoph.mueller@visus.uni-stuttgart.de">
<span style="color:blue">christoph.mueller@visus.uni-stuttgart.de</span></a></span></p>
<p class="MsoNormal"><span lang="DE" style="font-size:10.5pt">Telefon: (07 11) 685 - 88626
</span></p>
<p class="MsoNormal"><span lang="DE"> </span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>