<html><body>
<p><tt>Roland Dreier <rdreier@cisco.com> wrote on 11/16/2006 11:26:31 AM:<br>
<br>
> > What I have found in ehca driver, n! = t, does't mean it's empty. If poll<br>
> > again, there are still some packets in cq. IB_CQ_REPORT_mISSED_EVENTS most<br>
> > of the time reports 1. It relies on netif_rx_reschedule() returns0 to exit<br>
> > napi poll. That might be the reason in poll routine for a long time? I will<br>
> > rerun my test to use n! = 0 to see any difference here.<br>
> <br>
> Maybe there's an ehca bug in poll CQ? If n != t then it should mean<br>
> that the CQ was indeed drained. I would expect a missed event would<br>
> be rare, because it means a completion occurs between the last poll CQ<br>
> and the request notify, and that shouldn't be that common...<br>
> <br>
> My rough estimate is that even at a higher throughput than what you're<br>
> seeing, IPoIB should only generate ~ 500K completions/sec, which means<br>
> the average delay between completions is 2 microseconds. So I<br>
> wouldn't expect completions to hit the window between poll and request<br>
> notify that often.<br>
> <br>
> - R.<br>
</tt><br>
<tt>I have tried low_latency is 1 to disable TCP prequeue, the throughput was increased from 1XXMb/s to 4XXMb/s. If I delayed net_skb_receive() a little bit, I could get around 1700Mb/s. If I totally disable netif_rx_reschedule(), then there is no repoll and return 0, I could get around 2900Mb/s throughout without packet seeing out of order issues. I have tried to add a spin lock in ipoib_poll(). And I still see packets out of orders.</tt><br>
<br>
<tt>disable prequeue: 2XXMb/s to 4XXMb/s (packets out of order)</tt><br>
<tt>slowdown netif_receive_skb: 17XXMb/s (packets out of order)</tt><br>
<tt>don't handle missed event: 28XXMb/s (no packets out of order)</tt><br>
<tt>handler missed envent later: 7XXMb/s to 11XXMb/s (packets out of order)</tt><br>
<br>
<tt>Maybe it is ehca driver deliver packets much faster? Which makes me think user processes tcp backlogqueue, prequeue might be out of order? </tt><br>
<br>
<tt>Thanks</tt><br>
<tt>Shirley Ma</tt></body></html>