[ofw] Use of I/O completion port with Winverbs

[Fab Tillier]

Ah, I didn’t realize Mellanox included the headers in their redist, ok.

The docs for NDv2 are in the pipeline, but there’s quite a backlog for our tech writers so I don’t have an ETA unfortunately.  The combination of NDv1 and NDK docs should get you on the right path in the meantime, and you can always send me questions.

Provider discovery is largely the same as NDv1, though you call DllGetClassObject for the IND2Provider interface directly, rather than going through the IClassFactory interface.

You should be able to interop between ND and librdma code, as well as between Windows and Linux no problem.  If it doesn’t work then there’s a bug in the drivers somewhere.  Of course you’ll be limited by the capabilities of NDv2 (no remote atomic, only reliable connected, no immediate data, etc.)

-Fab

From: Christoph Müller [mailto:christoph.mueller at visus.uni-stuttgart.de]
Sent: Sunday, August 18, 2013 12:01 PM
To: Fab Tillier
Cc: 'ofw at lists.openfabrics.org'
Subject: AW: Use of I/O completion port with Winverbs

Hi Fab,

thanks. I already got access to the IDN2Provider on Friday - the headers are also included in Mellanox' distribution just as Tzachi said. The problem on my side was that I was looking for some import libraries. I finally found some NDv1 documentation on MSDN explaining how to get the root object via the Winsock provider catalogue. However, there does not seem to be any NDv2 documentation yet, but the differences seem to be subtle.

As I have only limited time, I will probably not be able to rewrite all of my code at once. I think that my NDv2 code will be able to talk to librdma peers as long as I use only basic functionality like remote read/write or inline send/receive. Is that correct?

Best regards,
Christoph

Von meinem Windows Phone gesendet
________________________________
Von: Fab Tillier<mailto:ftillier at microsoft.com>
Gesendet: ‎18.‎08.‎2013 07:38
An: Christoph Müller<mailto:christoph.mueller at visus.uni-stuttgart.de>
Cc: 'ofw at lists.openfabrics.org'<mailto:ofw at lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
Hi Christoph,

You can get the ND header files (included in the NetworkDirect DDK zip file) here:
http://www.microsoft.com/en-us/download/details.aspx?id=36043

There is no dependency on the HPC Pack - you just need drivers that support ND (ideally NDv2, which the Mellanox 4.40 release does).

Using ND should give you the greatest hardware independence too, though by design ND exposes the lowest common denominator functionality of iWARP and InfiniBand, so some functionality is not available (e.g. RDMA atomics).

Don’t hesitate to ask if you have questions.
-Fab

From: ofw-bounces at lists.openfabrics.org<mailto:ofw-bounces at lists.openfabrics.org> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Christoph Müller
Sent: Thursday, August 15, 2013 6:04 AM
To: Tzachi Dar; ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Subject: Re: [ofw] Use of I/O completion port with Winverbs

Thank you. I was not aware that the SDK including NDv2 is distributed with Mellanox’ drivers. In this case, I will give it a try.

Thanks again,
Christoph

Von: Tzachi Dar [mailto:tzachid at mellanox.com]
Gesendet: Donnerstag, 15. August 2013 14:43
An: Christoph Müller; ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs

You only need to download Mellanox drivers 4.40 for windows, and you will have nd support.

http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers

Thanks
Tzachi


From: Christoph Müller [mailto:christoph.mueller at visus.uni-stuttgart.de]
Sent: Thursday, August 15, 2013 3:32 PM
To: Tzachi Dar; ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Subject: AW: Use of I/O completion port with Winverbs

ndinstall is part of the HPC pack distribution? My main problem with ND is to find out how to get it...

Best regards,
Christoph

Von meinem Windows Phone gesendet
________________________________
Von: Tzachi Dar<mailto:tzachid at mellanox.com>
Gesendet: ‎15.‎08.‎2013 14:28
An: Christoph Müller<mailto:christoph.mueller at visus.uni-stuttgart.de>; ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
Please note that the windows APIs are changing from time to time and ND is probably the one that will stay longer since it is supported by MS.

I don’t think that you need to install the hpc pack for it. You simply need to run ndinstall and then you can start running nd programs.

Thanks
Tzachi

From: Christoph Müller [mailto:christoph.mueller at visus.uni-stuttgart.de]
Sent: Thursday, August 15, 2013 3:21 PM
To: Tzachi Dar; ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Subject: AW: Use of I/O completion port with Winverbs

Actually, I am in the process of deciding which API to use (although I have existing librdma code). NetworkDirect indeed looks promising, but as I understand, I need HPC Pack 2012 for that which I cannot install on our cluster in the short term.

Best regards,
Christoph


Von meinem Windows Phone gesendet
________________________________
Von: Tzachi Dar<mailto:tzachid at mellanox.com>
Gesendet: ‎15.‎08.‎2013 14:15
An: Christoph Müller<mailto:christoph.mueller at visus.uni-stuttgart.de>; ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Betreff: RE: Use of I/O completion port with Winverbs
can you write your application with Network Direct?
If so, there are overlapped objects there that you should be able to connect to the completion ports.

Thanks
Tzachi

From: ofw-bounces at lists.openfabrics.org<mailto:ofw-bounces at lists.openfabrics.org> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Christoph Müller
Sent: Thursday, August 15, 2013 3:10 PM
To: ofw at lists.openfabrics.org<mailto:ofw at lists.openfabrics.org>
Subject: [ofw] Use of I/O completion port with Winverbs

Hi all,

I was wondering whether it is possible to use CreateIoCompletionPort etc. with IWVConnectEndpoints. I can see that the implementation inherits a handle from CWVBase, but I do not see how I could use this information for creating an IOCP and, of course, I do not know whether it would work. Furthermore, I would prefer continuing to use the RDMA library, but I see absolutely no way to create an IOCP here. So me questions are:

Is it possible to use IOCPs with winverbs?
Is it possible to use IOCPs with the RDMA library?
Is there sample code for either of it?

Thanks in advance,
Christoph
--
Visualisierungsinstitut der Universität Stuttgart
Allmandring 19, 70569 Stuttgart
E-Mail: christoph.mueller at visus.uni-stuttgart.de<mailto:christoph.mueller at visus.uni-stuttgart.de>
Telefon: (07 11) 685 - 88626

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20130818/65da376b/attachment.html>