[openib-general] heads-up - ipoib NAPI

[Shirley Ma]

Hi, Eli,

Eli cohen <eli at dev.mellanox.co.il> wrote on 09/26/2006 11:35:26 PM:
> On Tue, 2006-09-26 at 21:34 -0700, Shirley Ma wrote:
>
> > NAPI patch moves ipoib poll from hardware interrupt context to softirq
> > context. It would reduce the hardware interrupts, reduce hardware
> > latency and induce some network latency. It might reduce cpu
> > utilization. But I still question about the BW improvement. I did see
> > various performance with the same test under the same condition.
> >
> When you open just one connection you can see around 10% of variations
> in BW measure. But then you don't utilize all the CPU power you have and
> you don't get to the threshold where NAPI becomes effective.
> Using multiple connections utilizes all CPUs in the system, increases
> send rate, and increases the chances of the receiver to poll CQEs up to
> its quota and be scheduled again without re-enabling interrupts.

Send rate shouldn't be limited by one connection. The cpu is much faster
than the link speed. I don't think multiple connections send rate is
increased than one connection. Do you have any data to show that?

When I monitored the CQEs, I didn't see too many CQEs in CQ for one
notification, and I don't think moving NAPI from hardware interrupt context
to softirq context would increase that number. Or the latency might cause
the number increased, I did see that number increased and performance
increased with some udelay in hardware interrupt polling mode. If you saw
the packets increased, how many packets did you see in both one hardware
interrupt poll and one NAPI poll?

You NAPI poll is driven either by receiver quota or any send CQE in CQ.
Have you tested UDP performance? any difference?

Thanks
Shirley Ma
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060927/c7ae1696/attachment.html>