[ofa-general] missed cq event

[Philip Frey1]

You are right. Thanks!

I have yet another issue:

Sometimes I get the following message in /var/log/messages of the local 
host:

post_qp_event - AE qpid 0x4e0 opcode 3 status 0x13 type 0 wrid.hi 0x0 
wrid.lo 0x65000000

I was looking for the status and opcode in the source and found that
opcode 3 means T3_SEND and status 0x13 means TPT_ERR_OUT_OF_RQE.
At the remote host I get and opcode 7 (T3_TERMINATE) and status 0x0 
(SUCCESS).

Clearly there is someone running out of Receive Queue Elements. The error 
occurred when
doing an ibv_post_send() at the local host. Is this a coincidence or does 
the local host
somehow know that there are not enough RQE's available at the remote host? 
In other words,
does the TPT_ERR_OUT_OF_RQE refer to the local or to the remote receive 
queue?

Many thanks,
 Philip

general-bounces at lists.openfabrics.org wrote on 09.06.2008 15:56:46:

> Philip Frey1 wrote:
> >
> > > > what is the correct way to get completions from the CQ without 
> > missing
> > > > any?
> > > > I am currently using two separate CQs for the SQ and for the RQ.
> > > > In pseudocode I do about the following to wait for a completion:
> > > >
> > > > ibv_get_cq_event(channel, &dst_cq, &ctx);        //blocking wait 
for
> > > > CQ event
> > > > <-- EVENT -->
> > > > ibv_req_notify_cq(dst_cq, 0);                        // request 
event
> > > > for next completion
> > > > ibv_poll_cq(dest_cq, 1, wc);                        // get the WC 
> > from
> > > > the CQ
> > > > ibv_ack_cq_events(dst_cq, 1);                        // ack the 
event
> > > >
> >
> > < snip >
> >
> > >
> > > you need to poll the cq until it is empty instead of just polling 
once.
> > >
> > >
> > > get_cq_event()
> > > req_notify_cq()
> > > do {
> > >     ne = poll_cq();
> > > } while (ne != 0);
> > > ack_cq_event()
> >
> > But what if I am just interested in the next work completion? Wouldn't 

> > this mean
> > that I have to build a queue in my application, that can hold 
> > completions in which
> > I am not yet interested? It sounds like a replication of the 
> > completion queue to me.
> > I was more looking for a way to do the 'single poll' on the completion 

> > without copying
> > anything into another queue.
> 
> You can process each polled CQE individually inside the loop, yes?  How 
> is that different from getting notified of each CQE insertion and 
> processing it then?
> 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
http://openib.org/mailman/listinfo/openib-general
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080609/d21d4992/attachment.html>