[openib-general] Re: IPOIB_NUM_WC question

[Michael S. Tsirkin]

Quoting r. Shirley Ma <xma at us.ibm.com>:
> Subject: Re: IPOIB_NUM_WC question
> 
> 
>     Shirley> Also I noticed that mthca sends IB_CQ_NEXT_COMP with
>    Shirley> completion entry==0 pretty often, in my netperf test,
>    Shirley> more than 30% time.
> 
> >I can't quite understand this.  What do you mean by sending IB_CQ_NEXT_COMP?
> 
> What I meant the IB_CQ_NEXT_COMP notification was received even if entry==0. There was no wc to poll out from the cq.
> 
> Thanks
> Shirley Ma

The reason could be that we poll CQ after doing arm (IB spec requires this).
So if a completion is generated while we poll the CQ, we get an extra event.

Mellanox hardware however does not have this requirement.
Could you test the following patch please? I'll be also interested
to know what is going on with overall performance after applying it.

If this helps a lot, I think we could add some capability bit to let
ULPs know they don't need to poll after arm.

Index: openib/drivers/infiniband/ulp/ipoib/ipoib_ib.c
===================================================================
--- openib/drivers/infiniband/ulp/ipoib/ipoib_ib.c	(revision 5588)
+++ openib/drivers/infiniband/ulp/ipoib/ipoib_ib.c	(working copy)
@@ -293,12 +293,12 @@ void ipoib_ib_completion(struct ib_cq *c
 	struct ipoib_dev_priv *priv = netdev_priv(dev);
 	int n, i;
 
-	ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 	do {
 		n = ib_poll_cq(cq, IPOIB_NUM_WC, priv->ibwc);
 		for (i = 0; i < n; ++i)
 			ipoib_ib_handle_wc(dev, priv->ibwc + i);
 	} while (n == IPOIB_NUM_WC);
+	ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 }
 
 static inline int post_send(struct ipoib_dev_priv *priv,

-- 
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies