[ofa-general] Re: IPoIB forwarding

[Michael S. Tsirkin]

> Quoting Bryan Lawver <lawver1 at llnl.gov>:
> Subject: IPoIB forwarding
> 
> I have a small test bed with 2 nodes with IB/OFED1.2/connected mode and a 
> third node which has IP only and is connected to one of the IB nodes.  In 
> between are DDR IB switch and 10GE IP switch.  The node with both IP and IB 
> interfaces is simply a IP router in this test setup.  The IB only node has 
> a subnet route to router node and the IP only node has a subnet route to 
> the router node.
> 
> When I launch an Iperf test from the IB (IPoIB) node to the IP node, I get 
> very good throughput with no tuning (7.5gbs).
> 
> When I launch from IP to the IB node, I get virtually no thorughput 
> (2.5mbs).  When I dropped the window size to 8k (iperf -w8k) the throughput 
> is 750mbs.
> 
> Any suggestions, ideas?

Some troubleshooting tips:

Are some packets lost on the router? Checking packet counters
might give you a clue.

Do you see some errors on one of the IB nodes?
Set debug_level=1 module parameter for ib_ipoib, and check
dmesg output while running the test.

-- 
MST