[ofw] How to block waiting for a CQE?

[Diego Guella]

I have used the pseudo-code below in a real application, and don't see any problem or high processor usage, even with realtime 
priority.


Thanks,
Diego



----- Original Message ----- 
From: "Diego Guella"
>
> ----- Original Message ----- 
>>From: "Tzachi Dar"
>>Ib rearm CQ will fire if there is an event after the last polled cqe.
>>That is, if there is a new cqe it will fire, even if it happens between
>>the poll
>>And the arm.
>>As a result, the code will work, and you don't have to wait for the
>>second event.
>
> Oh. This is what I was missing.
> I am using Mellanox hw now, so I shouldn't have problems.
>
>
>>[All said is true for Mellanox HW. I don't know what happens for other
>>hw vendors.]
> In ib_al.h:
> /****f* Access Layer/ib_rearm_cq
> * NAME
> * ib_rearm_cq
> *
> * DESCRIPTION
> * This indicates that the completion queue should notify the client when
> * the next completion is added.
> *
> * SYNOPSIS
> */
>
> ...
>
> * NOTES
> * This routine instructs the channel interface to invoke the completion
> * handler when the next completion queue entry is added to this CQ.
>
> (In fact, this is the behavior of ibv_req_notify_cq/ibv_get_cq_event on Linux).
> So if some other vendor implemented ib_rearm_cq using the strict behavior in ib_al.h, I will run into the problem I outlined.
> Same thing if Mellanox changes the behavior to conform to what is written in ib_al.h.
>
> Please look at the modified pseudocode below:
> -----
> // Initialization
> ib_status = ib_rearm_cq( ctx->scq, FALSE );
>
> // Loop
> while (1) {
> cl_status = cl_waitobj_wait_on( ctx->cq_waitobj, EVENT_NO_TIMEOUT, TRUE );
> ib_status = ib_rearm_cq( ctx->scq, FALSE );    // ib_rearm_cq moved before poll loop
>
> do {
>  ib_status = ib_poll_cq(ctx->scq, &p_wc_free, &p_wc_done);
>  if (ib_status == IB_SUCCESS)
>   handle_stuff();
> } while (ib_status == IB_SUCCESS );
>
> }
> -----
>
> This has the drawback that on Mellanox hw I will double the number of outer loops, but at least I have a spec-compliant behavior 
> from my side.
> What do you think about it? Do you see some problems in it?
>
>
> Thanks,
> Diego
>
>
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>
>