[ofw] post send in kernel mode

[Sean Hefty]

>I wish to discuss yet another option for RDS protocol:
>Connection establishment in user mode (use of the rdma_cm over Winverbs)
>and data xfer in kernel mode using post send operation.
>
>Basically QP and CQ can be obtained in kernel mode winverbs driver
>using WvQpAcquire(..) and WvCqAcquire() respectively.
>And post_send and poll_cq done using HCA verbs interface calls
>QP->pVerbs->post_send() and QP->pVerbs->poll_cq().
>
>Please let me know this kernel mode option for data xfer is feasible or
>am  I missing anything here.

I think the only functionality that's needed from userspace is the equivalent
rdma_resolve_addr/rdma_resolve_route.  (We kind of keep coming back to this.)
The actual connection could still be done in the kernel.

The basic way the stack works today is that librdmacm performs the address and
route resolution.  It then makes a 'Modify' call into the winverbs library to
set the address and path information, which sets the kernel data structures.
Not exactly ideal, but I don't know that this can cause any real harm.

If you're looking at a split model, then I would limit what's required from
userspace to obtaining the information.  Everything else should be doable from
the kernel.  This kind of gets us back to what in userspace provides the
information and through what sort of interface.

I guess a question to ask is would supporting this on 2008+ be enough?

- Sean