[ofa-general] [RFH] IPoIB retransmission when sending multiple WR's to device

[Krishna Kumar2]

(Request For Help)

Hi,

On the same topic that I wrote about earlier, I put debugs
in my code to store all skbs in bufferA when enqueing multiple
skbs, and store all skbs to bufferB just before doing post.
During post, I compare the two buffers to make sure that I am
not posting in the wrong order, and that never happens.

But I am getting a huge amount of retransmissions anyway,
and I don't understand why this happens. It is either the
IPoIB driver changes I made, or driver that is processing
multiple WR's, or firmware that is sending out of order.
I don't see high retransmission when I run over E1000 (had
ported E1000 driver too, just to make sure that the core
networking code is not putting skbs out of order, and
results confirms it is correct). The retransmission number
is around 200 vs 6000-10000 / sec for Original/New IPoIB
drivers. The result of retransmission is affecting when I
have high number of threads (like 64), but in small # of
threads, even with the retransmission, I am getting very
good improvement in throughput. But with 64 threads, I am
getting a fall from 270 MBPs to 250 MBPs.

Since I am really stuck for the last couple of weeks on this,
can someone help by taking a quick look at my code (attached
as just the functions that are changed), and/or suggest some
way to debug/fix this ?

Thanks,

- KK

(See attached file: CODE)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CODE
Type: application/octet-stream
Size: 10792 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070802/5f84aee3/attachment.obj>