[openib-general] Completion callback /teardown race

[Fabian Tillier]

Hi Michael,

On 9/19/06, Michael S. Tsirkin <mst at mellanox.co.il> wrote:
> Quoting r. Fabian Tillier <ftillier at silverstorm.com>:
> > > There are some differences in HCA behaviour with regard to
> > > ib_req_notify_cq.  Mellanox HCAs will provide a callback/interrupt if
> > > the CQ is not empty at this point (in which case the poll_cq's after the
> > > notify are optional).
> > >
> > > However the behaviour defined in the IBTA spec indicates that
> > > ib_req_notify_cq will cause a callback/interrupt only on the next CQE
> > > which arrives, hence to be portable the poll_cq loop after
> > > ib_req_notify_cq is necessary to cover any CQEs which arrived between
> > > the prior poll and the ib_req_notify_cq.
> >
> > I remember a while ago a mention that the behavior of the Mellanox
> > HCAs could be controlled in the firmware, so that they would follow
> > the IBTA spec defined behavior.
>
> There's a mistake here. Mellanox HCAs will generate an event upon
> ib_req_notify_cq only if new completions has arrived after the previous event
> has been reported.

Thanks for correcting me - I expected my memory to be a bit rusty.  In
this case, is there any benefit in polling before calling
ib_req_notify_cq?

> AFAIK this is IBTA spec compliant.

Yes, I believe it is too.  Do you know if there is any impact on
performance in doing the following for completion processing:

ib_req_notify_cq
poll_cq until empty

Thanks,

- Fab