[ofw] port RDS to OFW - request for input...

[Tzachi Dar]

The way I understand it, RDS is being used by user mode processes with a
socket like interface.
If that is indeed the case than I would also like to add another
possibility:

Implement the entire protocol in user mode.

There are a few advantages to this approach:
1) From the existing libraries perspective one can use ND and RDMA cm.
2) From the performance perspective this has advantage when it comes to
packet rate and latency (this is because one can send data and receive
it without going into the kernel). Achieving maximum BW should also not
be a problem with this approach. Handling interrupts should have
somewhat higher overhead.

This method works best when the number of processes using RDS on each
computer is not big. (If I remember correctly RDS opens one QP between
each two machines).
I realize that it is not that common to implement network code in the
user space, but this is one big advantage that IB gives.

As for the other questions that you have asked (assuming RDS is being
implemented in kernel).
For the data path there is no choice but to use IBAL (currently there
are no winverbs in the user).

As for using the winverbs.sys driver in kernel. Winverbs.sys only helps
the winverbs user mode with operations that are not in the data path. It
will not really be useful for use from the kernel, as you also need the
data path operations.

For creating the connections you can either use a service (this is the
windows term for user mode demon) or implement the cma in the kernel.
Please note that the CMA file in the kernel is not that big (~1000
lines) so porting it kernel shouldn't be that big of a problem. On the
other hand, CMA has it's own completion semantic... If you want to keep
it that might take some extra work.

I hope this helps.

Thanks
Tzachi

> -----Original Message-----
> From: ofw-bounces at lists.openfabrics.org 
> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Richard Frank
> Sent: Wednesday, July 22, 2009 7:22 PM
> To: ofw at lists.openfabrics.org
> Subject: [ofw] port RDS to OFW - request for input...
> 
> We are investigating what is required to port the RDS driver to OFW..
> 
> RDS is a kernel mode driver which uses CMA (rdmacm) interfaces.
> 
> A general question we are thinking about is whether or not 
> there is a requirement for a kernel mode cma (rmda_cm) 
> library / driver for use by kernel mode clients in Windows.
> 
> We are looking for input - to see if there is a general 
> requirement for this kernel rdma_cm lib and how to build it 
> (what interface should look like (direct call / IRP)) - or - 
> if we should consider alternatives which leverage existing 
> kernel interfaces / drivers.
> 
> Some alternatives are:
> 
> a) leverage existing wvdriver.sys - calling it directly from 
> kernel (creating IRPs, etc) and feeding it with required 
> information to complete address translation, etc..
> possibly using kernel mode sockets interfaces..
> 
> b) somehow leveraging the existing user mode lib rdma cm 
> (winverbs) via a user mode daemon which handles requests to 
> perform connection management operations from kernel mode drivers..
> 
> c) work directly with IBAL.. etc..
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>