[openib-general] question about poll_cq()

[Caitlin Bestler]

> -----Original Message-----
> From: openib-general-bounces at openib.org 
> [mailto:openib-general-bounces at openib.org] On Behalf Of Joerg Zinke
> Sent: Tuesday, October 25, 2005 2:40 AM
> To: openib-general at openib.org
> Subject: [openib-general] question about poll_cq()
> 
> hi,
> 
> i hope it's ok to ask this question here.
> i just want to know more about poll_cq().
> the standard seems to define only the input/ouput-params.
> after reading the code i figured out that there is some kind 
> of "mapping" from libibverbs to the device specific "plugin" (mthca).
> so it looks like that the ibv_poll_cq() call from the userspace (e.g.
> ibv_xx_pingpong) finally ends in a call of: mthca_poll_cq() 
> in mthca_cq.c
> 
> so my question is: can someone give me a brief summary what 
> mthca_poll_cq (or mthca_poll_one()) is really doing?
> i want to know how polling the completion queue really works, 
> there must be some kind of low-level acknowledge/response- 
> message if a new entry enters the cq?
> where is the completion queue really located (on a rdma 
> operation) - local or remote?
> 

Understanding how a given device implements poll_cq is 
legitimate if the purpose is debugging and/or understanding
memory/bus utilization. However it is one of those that you
SHOULD NOT know when righting code that uses poll_cq.

What will vary over time and model is where the work
completions are stored (in device and/or host memory)
how they are formatted and whether they are self contained
or reference other data (such as the original work request).
The work completion that you get from a successful poll_cq
may never have existed as that sequence of bytes until you
made the call to poll_cq.

If these things were not deliberately undefined then there
would be no need for both ibv_poll_cq and device specific methods.

For a discussion on what you can assume about a CQ across all
devices I'd suggest reviewing the IBTA and/or RDMAC verbs. But
basically, about all that is guaranteed is that it is a reliable
ordered queued that the Consumer MUST provision adequately so as
to avoid overflows.