[ofw] port RDS to OFW - request for input...

[Smith, Stan]

Richard Frank wrote:
> We are investigating what is required to port the RDS driver to OFW..
>
> RDS is a kernel mode driver which uses CMA (rdmacm) interfaces.
>
> A general question we are thinking about is whether or not there is a
> requirement for a kernel mode cma (rmda_cm) library / driver for use
> by kernel mode clients in Windows.
>
> We are looking for input - to see if there is a general requirement
> for this
> kernel rdma_cm lib and how to build it (what interface should look
> like (direct call / IRP)) - or - if we should consider alternatives
> which leverage existing kernel interfaces / drivers.
>
> Some alternatives are:
>
> a) leverage existing wvdriver.sys - calling it directly from kernel
> (creating IRPs, etc)
> and feeding it with required information to complete address
> translation, etc..
> possibly using kernel mode sockets interfaces..
>
> b) somehow leveraging the existing user mode lib rdma cm (winverbs)
> via a user mode daemon which handles requests to perform connection
> management operations from kernel mode drivers..
>
> c) work directly with IBAL.. etc..

<Notice the 'Brain Storming' stamp - proceed with caution>

>From the conversation so far a Windows RDS in the kernel implementation wants the following OFED kernel interfaces:
  FMR - Fast kernel memory registration.
  rdma_cm - ala Linux kernel mode interfaces
  ibverbs - Linux kernel mode interfaces (mainly post send/recv?)

Kernel mode IBAL provides FMR, IBAL-CM & IBAL-IB-verbs.

Assuming the user-mode RDS client opens a socket for an AF_RDS protocol family, then makes RDS (somewhat like UD) data transfer calls; the kernel mode RDS driver will receives IRP's containing control and possibly data [data proper or user-mode data pointer(s) scatter/gather?].

Taking Sean's divide-n-conquer approach let's talk about kernel shim layers.

1) How hard/reasonable would it be to provide an ibverbs interface layer over IBAL verbs; given it's RDS specific and targeted at data transfer (minimal subset)?

2) Is it possible/reasonable to construct an rdma_cm interface layer over IBAL-CM? How different are the semantics?


Question:  Are #1 & #2 any easier, faster or more sane than building kernel mode winverbs interfaces for FMR, rdma_cm and ibverbs-data-xfer?

OK, here comes the broad stroke brush.... (SWAG)

Building winverbs interfaces:
  FMR and ibverbs prototype development: 1 head 12 weeks; kind of hard to test without a CM?
  The 'real' issue is rdma_cm - building all the QP 0+1 MAD connection mgmt handing and speaking of MAD handling, IBAL, winverbs; who is in charge of MAD processing?  SWAG - 1 head 24 weeks to a working prototype.

Shimming IBAL -
  FMR and ibverbs interfaces: 1 head 8 weeks
  rdma_cm IF over IBAL-CM: 1 head 12 weeks?

Port kernel RDS to IBAL interfaces (replace rdam_cm & ibverbs IF with IBAL IFs) 6 - 8 months??

BTW, when I speak of heads - anyone of the following would do nicely.

Sean, would you like to explain how whacked all this is?  :-)
Fab or Tzachi ?

Flame suit zipped and ready.....


OFED
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw