[openib-general] ib_req_ncomp_notif in core_ layer

[Michael S. Tsirkin]

Hello!
Quoting r. Fab Tillier (ftillier at infiniconsys.com) "RE: [openib-general] ib_req_ncomp_notif in core_ layer":
> > From: Sean Hefty [mailto:mshefty at ichips.intel.com]
> > Sent: Wednesday, August 11, 2004 2:52 PM
> > 
> > On Thu, 12 Aug 2004 00:23:40 +0300
> > "Michael S. Tsirkin" <mst at mellanox.co.il> wrote:
> > 
> > > Currently some things are unclear:
> > >
> > > If you get an event and do req_comp_notif
> > > immediately, without polling - do you expect to get an event
> > immediately?
> > 
> > The call for ib_req_notify_cq() is intended to map to the semantics
> > mentioned in the spec.  For the example mentioned, calling
> > ib_req_notify_cq() without polling arms the CQ, but will not generate an
> > event until a new completion of the specified type is added to the CQ.
> 
> True, but as has been pointed out before, Tavor by default will generate a
> new CQ event if any completions are left unreaped in the CQ (or something to
> that effect).  See
> http://openib.org/pipermail/openib-general/2004-June/003099.html.  So
> rearming without polling *will* generate a CQ event if you're using Mellanox
> HCAs.

Nope, sorry. Tavor will generate an event if completion are generated
*after the event was generated*.

So 
   arm
   completion
   event
   arm    <-- no event

But
   
   arm
   completion
   event
   completion
   arm    <-- event

Thus you can do
   poll -> rearm
and also
   rearm -> poll

and get the same result without races.


> > 
> > > What currently happends as far as I can see, is:
> > > for req_comp_notif you dont, for req_ncomp_notif you do.
> > > For n=1 the behaviour of req_ncomp_notif probably does not
> > > make sence ...
> > >
> > > If you dont for req_ncomp_notif, what would you expect?
> > 
> > ib_req_n_notify_cq() is outside of the spec, so needs to be defined.  My
> > expectation would be that it behaves similar to ib_req_notify_cq(), in
> > that it generates an event after wc_cnt (or one solicited) *new*
> > completions are added to the CQ.
> 
> We're hitting the whole "should CQ rearm be spec compliant" issue again.  My
> understanding was that we were going for non-spec compliance because that's
> how the current Mellanox HCA is implemented (at least in the FW).  So to
> poll, a user does poll->rearm, not rearm->poll.

Thats not what, say, ip over ib currently does, as far as I can see.

> Either way, both rearm calls need to have the same general semantics, and
> the expected behavior needs to be clear so that future HCA HW and SW can be
> implemented "properly".

Either way this would have to be documented.
What do others here think?

MST