[Openib-windows] CQ event notification in user-mode

[Fab Tillier]

> From: Sean Hefty [mailto:sean.hefty at intel.com]
> Sent: Wednesday, August 17, 2005 2:19 PM
> 
> >True, but then do you apply the same logic to connection end points?  Should
> >each CEP have an underlying file so that the user can implement totally
> >different I/O completion processing on a per-connection basis?  I'm weary of
> >adding in useless flexibility at the cost of scalability.  I just don't quite
> >know where to draw the line.
> 
> I would treat completion processing separately from connection establishment.
> IMO, connection establishment doesn't need to be as flexible.  (Arlin may
> disagree with me here...)  Most apps would probably be happy to connect using
> host names or IP addresses.  With CQs, there's a way for an app to improve
> scalability if 1-2 CQs per QP consumes too many software resources - not so
> with
> CEPs.
> 
> So, I guess my vote is to ignore the potentially scalability issues of mapping
> one CQ to one file, then pick the best alternative.

When doing async I/O, the user has to provide an OVERLAPPED structure that
remains valid for the duration of the I/O.  Since the user has to allocate that,
it's pretty easy to embed that into a larger structure in such a way that the
OVERLAPPED structure can get the user back to their object (containing record
kind of thing).  I'm leaning towards having the key in the completion port
distinguish between different types of completions (CM-related, SA-related, CQ
completion related, and affiliated/unaffiliated HW events).  The key can be used
as an initial triage mechanism, and then the overlapped provides the rest.  The
key should give enough information to allow a user to cast back to the
appropriate structure given the OVERLAPPED structure.

This approach definitely would make the implementation easier, and I think the
loss in flexibility is minimal (and reasonable).

Sound reasonable?

- Fab