[openib-general] [IPoIB] executing iperf over IPoIB causes to multicast (IP) packets to be recieved out-of-order

[Dotan Barak]

Hi.

when executing iperf over IPoIB with multicast IP packets, there are some packets that being received out of order.


Here are my machine attributes:
*************************************************************
Host Architecture : x86_64
Linux Distribution: Fedora Core release 4 (Stentz)
Kernel Version    : 2.6.11-1.1369_FC4smp
Memory size       : 4071672 kB
Driver Version    : openib_gen2-20060524-1700 (REV=7460)
HCA ID(s)         : mthca0
HCA model(s)      : 25208
FW version(s)     : 4.7.600
Board(s)          : MT_00A0010001
*************************************************************

here are the iperf command line:
sender command:    ./iperf -c 10.4.3.86 -u -T 3 -t 400 -i 2 -b 1000M -l 100
receiver command: ./iperf -s -u -B 10.4.3.86 -i 2

here is the output of the receiver execution:

Execute the receiver over IPoIB
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 11.4.3.86
Receiving 1470 byte datagrams
UDP buffer size:  132 KByte (default)
------------------------------------------------------------
[  3] local 11.4.3.86 port 5001 connected with 11.4.3.87 port 33334
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[  3]  0.0- 2.0 sec  34.3 MBytes   144 Mbits/sec  0.003 ms  232/359959 (0.064%)
[  3]  0.0- 2.0 sec  45 datagrams received out-of-order
[  3]  2.0- 4.0 sec  34.3 MBytes   144 Mbits/sec  0.004 ms    0/359391 (0%)
[  3]  2.0- 4.0 sec  43 datagrams received out-of-order
[  3]  4.0- 6.0 sec  34.3 MBytes   144 Mbits/sec  0.003 ms    0/359470 (0%)
[  3]  4.0- 6.0 sec  31 datagrams received out-of-order
[  3]  6.0- 8.0 sec  34.3 MBytes   144 Mbits/sec  0.003 ms    0/359195 (0%)
[  3]  6.0- 8.0 sec  38 datagrams received out-of-order
[  3]  8.0-10.0 sec  34.3 MBytes   144 Mbits/sec  0.003 ms    0/359379 (0%)
[  3]  8.0-10.0 sec  48 datagrams received out-of-order
Waiting for server threads to complete. Interrupt again to force quit.
[  3]  0.0-21.2 sec   364 MBytes   144 Mbits/sec  0.005 ms  232/3813508 (0.0061%)
[  3]  0.0-21.2 sec  595 datagrams received out-of-order

Execute the receiver over IP
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 10.4.3.86
Receiving 1470 byte datagrams
UDP buffer size:  132 KByte (default)
------------------------------------------------------------
[  3] local 10.4.3.86 port 5001 connected with 10.4.3.87 port 33334
[ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total Datagrams
[  3]  0.0- 2.0 sec  14.4 MBytes  60.2 Mbits/sec  0.034 ms  355/150976 (0.24%)
[  3]  2.0- 4.0 sec  14.4 MBytes  60.2 Mbits/sec  0.010 ms    0/150599 (0%)
[  3]  4.0- 6.0 sec  14.4 MBytes  60.2 Mbits/sec  0.012 ms    0/150597 (0%)
[  3]  6.0- 8.0 sec  14.4 MBytes  60.2 Mbits/sec  0.011 ms    0/150599 (0%)
[  3]  8.0-10.0 sec  14.4 MBytes  60.2 Mbits/sec  0.009 ms    0/150598 (0%)
[  3] 10.0-12.0 sec  14.4 MBytes  60.2 Mbits/sec  0.010 ms    0/150584 (0%)
[  3] 12.0-14.0 sec  14.4 MBytes  60.2 Mbits/sec  0.009 ms    0/150583 (0%)
[  3] 14.0-16.0 sec  14.4 MBytes  60.2 Mbits/sec  0.019 ms    0/150601 (0%)
[  3] 16.0-18.0 sec  14.4 MBytes  60.2 Mbits/sec  0.009 ms    0/150576 (0%)
[  3] 18.0-20.0 sec  14.4 MBytes  60.2 Mbits/sec  0.021 ms    0/150583 (0%)
[  3] 20.0-22.0 sec  14.4 MBytes  60.2 Mbits/sec  0.010 ms    0/150591 (0%)
Waiting for server threads to complete. Interrupt again to force quit.
[  3] 22.0-24.0 sec  14.4 MBytes  60.2 Mbits/sec  0.028 ms    0/150582 (0%)


did anyone notice this issue before?

thanks
Dotan