[ofw] How to block waiting for a CQE?

[Tzachi Dar]

Ib rearm CQ will fire if there is an event after the last polled cqe.
That is, if there is a new cqe it will fire, even if it happens between
the poll
And the arm.
As a result, the code will work, and you don't have to wait for the
second event.

[All said is true for Mellanox HW. I don't know what happens for other
hw vendors.]


Thanks
Tzachi


> -----Original Message-----
> From: Diego Guella [mailto:diego.guella at sircomtech.com] 
> Sent: Thursday, February 19, 2009 12:47 PM
> To: Tzachi Dar; ofw at lists.openfabrics.org
> Subject: Re: [ofw] How to block waiting for a CQE?
> 
> 
> ----- Original Message ----- 
> >From: "Tzachi Dar" <tzachid at mellanox.co.il>
> >To: "Diego Guella" <diego.guella at sircomtech.com>; 
> ><ofw at lists.openfabrics.org>
> >Sent: Wednesday, February 18, 2009 5:08 PM
> >Subject: RE: [ofw] How to block waiting for a CQE?
> >
> 
> >Hi,
> >
> >You can get the example doing an svn co from the addresses:
> >svn://openib.tc.cornell.edu/gen1/trunk
> Thanks, I got the example and looked at it. But unfortunately 
> I still have some doubts :) Please see below.
> 
> >From the practical stand point you can use cl_waitobj_wait_on in as 
> >well as WaitForSingleObject.
> >To work more correctly from the standpoint of software 
> engneering you 
> >should call cl_waitobj_wait_on. This is because this is the 
> "standard"
> >way that ibal is using.
> >If you look at the implmentation you will see that 
> cl_waitobj_wait_on 
> >is being implmeanted by WaitForSingleObject.
> Ok, I supposed that. I asked because in the future I could 
> have the need to WaitForMultipleObjects.
> 
> 
> >As for the code that you have suggested. I don't think that it will 
> >work.
> >The main reason is that you don't do arm before you wait for 
> >notification.
> >I guess that this might work since after the arming is done, 
> you will 
> >get an event that will live your event signaled. This will trigger 
> >another loop of arming and polling which should work but is probably 
> >not needed.
> 
> This is the simplified pseudo-code from 
> tools\perftests\user\send_bw\send_bw.c:
> -----
> // Initialization
> ib_status = ib_rearm_cq( ctx->scq, FALSE );
> 
> // Loop
> while (1) {
>  cl_status = cl_waitobj_wait_on( ctx->cq_waitobj, 
> EVENT_NO_TIMEOUT, TRUE );
> 
>  do {
>   ib_status = ib_poll_cq(ctx->scq, &p_wc_free, &p_wc_done);
>   if (ib_status == IB_SUCCESS)
>    handle_stuff();
>  } while (ib_status == IB_SUCCESS );
> 
> // <--- What happens if a new CQE appears here?
> 
>  ib_status = ib_rearm_cq( ctx->scq, FALSE ); }
> 
> -----
> 
> I think that if a new CQE appears in the place I indicated, I 
> will have to wait until another CQE will appear, then I will 
> find 2 CQE in the inner poll loop.
> Am I misunderstanding something?
> 
> 
> Thanks,
> Diego
> 
> 
>