[ofa-general] IPoIB forwarding

[Bryan Lawver]

I have a small test bed with 2 nodes with IB/OFED1.2/connected mode and a 
third node which has IP only and is connected to one of the IB nodes.  In 
between are DDR IB switch and 10GE IP switch.  The node with both IP and IB 
interfaces is simply a IP router in this test setup.  The IB only node has 
a subnet route to router node and the IP only node has a subnet route to 
the router node.

When I launch an Iperf test from the IB (IPoIB) node to the IP node, I get 
very good throughput with no tuning (7.5gbs).

When I launch from IP to the IB node, I get virtually no thorughput 
(2.5mbs).  When I dropped the window size to 8k (iperf -w8k) the throughput 
is 750mbs.

Any suggestions, ideas?

thanks,
bryan lawver
llnl

[  4] local 192.168.120.3 port 33418 connected with 172.16.13.2 port 5001
[  5] local 192.168.120.3 port 5001 connected with 172.16.13.2 port 1032
[  4]  0.0-10.0 sec  4.72 GBytes  4.06 Gbits/sec
[  5]  0.0-10.3 sec  1.69 MBytes  1.38 Mbits/sec