<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7226.0">
<TITLE>Re: [openib-general] Mellanox request CQ notification _really_ compliant?</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>Roland,<BR>
<BR>
The spec does not mandate the mechanism by which the cqe is made available to the consumer. And other than ruling the order at which cqes are generated, there are no guarantees re the time it will take until a cqe is made visible to the consumer (unless there was a completion event generated).<BR>
Given that, the app in your example can not assume that the bind completion is already in the cq, at least not to the extent that a single call to pollforcompletion would be guaranteed to successfully retrieve it. So the app can not really perceive the difference.<BR>
<BR>
Thanks,<BR>
<BR>
Diego<BR>
<BR>
<BR>
----- Original Message -----<BR>
From: openib-general-bounces@openib.org <openib-general-bounces@openib.org><BR>
To: openib-general@openib.org <openib-general@openib.org><BR>
Sent: Mon Nov 27 15:41:14 2006<BR>
Subject: [openib-general] Mellanox request CQ notification _really_ compliant?<BR>
<BR>
So I was reading the IB spec and I noticed the following compliance<BR>
statement:<BR>
<BR>
C10-113: A CI shall not generate a Completion Event for existing<BR>
CQ entries on the specified CQ at the time the completion<BR>
notification request is registered.<BR>
<BR>
And I wonder if the well-known Mellanox behavior of generating an<BR>
event for existing CQEs is really compliant. (Don't get me wrong --<BR>
the behavior is so useful that if it isn't compliant, then I think the<BR>
spec is what needs correction)<BR>
<BR>
It seems to me that I could concoct a situation where a consumer could<BR>
actually prove that this compliance statement has been violated. For<BR>
example I think the following example works:<BR>
<BR>
- create 2 CQs, A and B -- they are now known to be empty since they<BR>
are freshly created.<BR>
- create 2 QPs, X and Y, and connect them to each other (loopback).<BR>
Use CQ A for QP X and CQ B for QP Y.<BR>
- post a receive on QP X.<BR>
- post a bind memory window operation on QP Y.<BR>
- post an unsignaled send on QP Y.<BR>
- poll CQ A until a receive completion appears (which it will, since<BR>
the send from QP Y will be received on QP X)<BR>
- We now know CQ B has exactly one completion, namely the bind<BR>
completion, since the send was executed and C10-98.2.1 says that<BR>
work requests after a bind may not begin execution until after the<BR>
bind completes.<BR>
- Request notification on CQ B. We should never get an event, but we<BR>
will on Mellanox HCAs.<BR>
<BR>
I guess one way around this is to say that a work request may be<BR>
"completed" before the work completion has been entered in the<BR>
corresponding completion queue. I don't see a rigorous definition of<BR>
what it means for a request to be completed in the IB spec.<BR>
<BR>
- R.<BR>
<BR>
<BR>
<BR>
_______________________________________________<BR>
openib-general mailing list<BR>
openib-general@openib.org<BR>
<A HREF="http://openib.org/mailman/listinfo/openib-general">http://openib.org/mailman/listinfo/openib-general</A><BR>
<BR>
To unsubscribe, please visit <A HREF="http://openib.org/mailman/listinfo/openib-general">http://openib.org/mailman/listinfo/openib-general</A><BR>
<BR>
</FONT>
</P>
</BODY>
</HTML>