[ofa-general] Re: IPOIB NAPI

[Roland Dreier]

 > > poll-cq
 > > notify-cq, if missed_event && netif_rx_reschedule()
 > > return 1
 > > 
 > > vs.
 > > poll-cq,
 > > notify-cq, if missed_event && netif_rx_reschedule()
 > > poll again
 > > return 0
 > > 
 > > It seems ehca delivering packet much faster than other HCAs. So poll again
 > > would stay in the loop for many many times. So the above changes doesn't impact
 > > other HCAs, I would recommand it. I saw same implementations on other ethernet
 > > drivers.
 > 
 > I have not benchmarked this, but actually the "return 1" version makes sense to
 > me too: since a new completion was observed after notify-cq, we likely currently
 > have HCA writing new completions into the CQ at a high rate, so it makes sense
 > to delay polling by a few cycles, and reduce the number of interrupts in this
 > way.

Yes, this does make sense.  It's kind of a cheap way to hold off a
little and try to get more work to do before we poll again, without
trying something more complex that is likely not to work well.

So just to confirm, the version that everyone likes is:

int ipoib_poll(struct net_device *dev, int *budget)
{
	struct ipoib_dev_priv *priv = netdev_priv(dev);
	int max = min(*budget, dev->quota);
	int done;
	int t;
	int empty;
	int n, i;

repoll:
	done  = 0;
	empty = 0;

	while (max) {
		t = min(IPOIB_NUM_WC, max);
		n = ib_poll_cq(priv->cq, t, priv->ibwc);

		for (i = 0; i < n; ++i) {
			// [completion handling deleted for clarity]
		}

		if (n != t) {
			empty = 1;
			break;
		}
	}

	dev->quota -= done;
	*budget    -= done;

	if (empty) {
		netif_rx_complete(dev);
		if (unlikely(ib_req_notify_cq(priv->cq,
					      IB_CQ_NEXT_COMP |
					      IB_CQ_REPORT_MISSED_EVENTS))) {
			netif_rx_reschedule(dev, 0);
			return 1;
		}

		return 0;
	}

	return 1;
}

with the significant part being that we return 1 instead of repolling
after we reschedule the polling routine.

 - R.