[openib-general] sharing qp between user and kernel

[Steve Wise]

On Thu, 2007-02-08 at 10:24 -0500, Pete Wyckoff wrote:
> rdreier at cisco.com wrote on Wed, 07 Feb 2007 15:50 -0800:
> >     Pete> Before I dig into this anymore, do you expect this to work?
> >     Pete> Are there fundamental problems with QP sharing between user
> >     Pete> and kernel?  It would sure be nice not to have to stick the
> >     Pete> connection management aspects into the kernel.
> > 
> > No, I wouldn't expect this to work.  At first glance at least, yes,
> > there are fundamental problems.  Sharing a QP between user and
> > kernelspace, where userspace is doing full kernel bypass (as eg mthca
> > does -- there are NO system calls when doing post work request, poll
> > CQ and request CQ notification operations), seems like a huge
> > problem.  I don't see any way that the kernel can keep a consistent
> > view of the QP state unless userspace has to call into the kernel for
> > every operation, which would kill performance.
> 
> My hope was not to allow full QP sharing between user and kernel,
> but just a limited interface to "send this kernel data now".  It
> requires that the kernel register some physical memory, and submit
> the work requests.  Perhaps the kernel can invoke the equivalent of
> the userspace post function instead of trying to use the kernel API
> for sending.
> 

You could map the kernel data into the user process and then have the
user process post the WR.  But the user process would have to have that
memory registered as part of a MR to post it.  It could be done though.
So basically instead of sharing QP, give your kernel module access
memory from a registered MR.  The kernel module can produce the data in
that memory then tell the user process to post the WR...
 

Steve