[openib-general] uVerbs and multiple cq_event support?

[Libor Michalek]

On Fri, Jul 15, 2005 at 01:26:11PM -0700, Arlin Davis wrote:
> Hi Roland,
> 
> I have a question about uverbs cq_events. Currently, uverbs has one 
> event (fd) handle that processes all CQ events. You mentioned earlier  
> that you would possibly add support for more CQ events. Are you thinking 
> one per CQ, or limiting to a small number like one per processor? If the 
> thinking is NOT one per CQ then it may be somewhat restricitive to user 
> applications that have multple connections with many CQs to process.
> 
> For instance, with uDAPL there are user wait objects (CQ_WAIT_OBJECT) 
> that can be mapped directly to these CQ events. This enables the user 
> thread to wait and wake up directly from the kernel CQ event processing 
> which is very efficicent. However, this can only be done if verbs 
> provides an event mechanism per CQ. Otherwise, I have to process the 
> events in a separate thread and in turn process the user's event 
> independent of the CQ event.
>
> If the thinking is one per CQ then is fd's the way to go since we can 
> have 1000's of CQs?

  An fd per CQ is definetly not the way to go for this reason. I don't
think that the wait object would need to map to an individual fd for
each CQ. The fd should become ready for read when there's a CQ event
pending and a read of the fd will tell you which CQ it was that had
the event. Only the wait object which represents that CQ needs to
notify the consumer, the rest can go back to waiting or continue waiting
depending how you implement the wait and locking around the fd. The
point being is that you do not need another thread.

  I'm just looking at the ABI, so I'm not sure which function
you should use to read the fd and get the info...

-Libor