[openib-general] Mellanox request CQ notification _really_ compliant?

[Diego Crupnicoff]

Roland,

The spec does not mandate the mechanism by which the cqe is made available to the consumer. And other than ruling the order at which cqes are generated, there are no guarantees re the time it will take until a cqe is made visible to the consumer (unless there was a completion event generated). 
Given that, the app in your example can not assume that the bind completion is already in the cq, at least not to the extent that a single call to pollforcompletion would be guaranteed to successfully retrieve it. So the app can not really perceive the difference. 

Thanks,

Diego


----- Original Message -----
From: openib-general-bounces at openib.org <openib-general-bounces at openib.org>
To: openib-general at openib.org <openib-general at openib.org>
Sent: Mon Nov 27 15:41:14 2006
Subject: [openib-general] Mellanox request CQ notification _really_ compliant?

So I was reading the IB spec and I noticed the following compliance
statement:

    C10-113: A CI shall not generate a Completion Event for existing
    CQ entries on the specified CQ at the time the completion
    notification request is registered.

And I wonder if the well-known Mellanox behavior of generating an
event for existing CQEs is really compliant.  (Don't get me wrong --
the behavior is so useful that if it isn't compliant, then I think the
spec is what needs correction)

It seems to me that I could concoct a situation where a consumer could
actually prove that this compliance statement has been violated.  For
example I think the following example works:

 - create 2 CQs, A and B -- they are now known to be empty since they
   are freshly created.
 - create 2 QPs, X and Y, and connect them to each other (loopback).
   Use CQ A for QP X and CQ B for QP Y.
 - post a receive on QP X.
 - post a bind memory window operation on QP Y.
 - post an unsignaled send on QP Y.
 - poll CQ A until a receive completion appears (which it will, since
   the send from QP Y will be received on QP X)
 - We now know CQ B has exactly one completion, namely the bind
   completion, since the send was executed and C10-98.2.1 says that
   work requests after a bind may not begin execution until after the
   bind completes.
 - Request notification on CQ B.  We should never get an event, but we
   will on Mellanox HCAs.

I guess one way around this is to say that a work request may be
"completed" before the work completion has been entered in the
corresponding completion queue.  I don't see a rigorous definition of
what it means for a request to be completed in the IB spec.

 - R.

 

_______________________________________________
openib-general mailing list
openib-general at openib.org
http://openib.org/mailman/listinfo/openib-general

To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20061127/a7cbbbe8/attachment.html>