[ofw] userspace rearm_cq

[Fab Tillier]

> -----Original Message-----
> From: Sean Hefty
> Sent: Monday, June 16, 2008 11:59 AM
>
> Can anyone explain the behavior of ib_rearm_cq() wrt issuing an IOCTL
> versus just calling the uvp?  Between the macros and function pointers,
> it's difficult to understand when exactly an IOCTL issues the rearm in
> the kernel, versus a direct call just being made to the uvp.

The way rearm works in IBAL is the call goes to the UVP and the UVP just rings the CQ's doorbell.  No kernel transition is explicit as part of this call.

For callback driven completions, IBAL internally issues an IOCTL to get CQ notifications, and these notifications (and subsequent callbacks) are processed on IBAL's internal thread pool.

For event driven completions, the event associated with the CQ is mapped to the kernel when the CQ is created, and signaled whenever a completion notification is received.  This wakes up whatever threads are waiting (depending on the type of event selected by the user).

For Network Direct, the UVP is not involved, the client's OVERLAPPED is used in an IOCTL to the kernel, where the CQ doorbell is rung and the IRP queued.  When the notification event occurs, the IRP is completed from the CQ's DISPATCH_LEVEL callback.

-Fab