[openib-general] Lustre Network Driver - KDAPL or verbs?

[Fab Tillier]

> From: Peter J. Braam [mailto:braam at clusterfs.com]
> Sent: Sunday, October 09, 2005 2:18 PM
> 
> Cluster File Systems, Inc and its customers have been wondering if the Lustre
> Network Driver (LND) for OpenIb gen2, which we will begin to develop during
> the coming months, should be based on kdapl or verbs.
> 
> The driver we plan to develop should strive to address several goals:
>  - high reliability and performance
>  - allow interoperability between user and kernel level
>  - allow interoperability, or better, portability among different operating
> systems (Linux, OS X, Windows, Solaris)
>  - be suitable for inclusion in the Linux kernel

I think that suitability for inclusion in the Linux kernel is going to be
mutually exclusive with portability between different operating systems.  If you
want to be in the Linux kernel, you need to be a native Linux driver, and not
use any sorts of abstraction layers.  Feedback to date on abstraction layers has
been consistently clear that they will not be tolerated in the kernel.

With the ongoing work to support both IB and iWarp devices under the OpenIB
verbs, I think coding directly to verbs would be just fine.  You'll likely want
to use the higher level CM abstraction being developed now for establishing
connections in a transport neutral manner, but the verbs themselves should be
the same.  Others more closely involved can likely give you better guidance.

With all this said, I'm personally interested to see a cluster file system on
top of the OpenIB Windows stack, and since kDAPL doesn't exist in Windows at the
moment, interfacing to native verbs would be my preference.  There really aren't
that many differences in verbs, though Windows will likely make you deal with
more things asynchronously depending on your IRQL.  I'd be happy to field
specific questions about Windows on the openib-windows mailing list if you have
them.

Cheers,

- Fab