[ofa-general] Re: IPoIB forwarding

[Bryan Lawver]

Thanks for the input.  I had found the debug switch and it pointed out some 
issues. Drops at the routing node are many.  It seems to work at debug rate 
so I hope it is not a buffer pool issue.

At 05:46 AM 4/25/2007, Michael S. Tsirkin wrote:
> > Quoting Bryan Lawver <lawver1 at llnl.gov>:
> > Subject: IPoIB forwarding
> >
> > I have a small test bed with 2 nodes with IB/OFED1.2/connected mode and a
> > third node which has IP only and is connected to one of the IB nodes.  In
> > between are DDR IB switch and 10GE IP switch.  The node with both IP 
> and IB
> > interfaces is simply a IP router in this test setup.  The IB only node has
> > a subnet route to router node and the IP only node has a subnet route to
> > the router node.
> >
> > When I launch an Iperf test from the IB (IPoIB) node to the IP node, I get
> > very good throughput with no tuning (7.5gbs).
> >
> > When I launch from IP to the IB node, I get virtually no thorughput
> > (2.5mbs).  When I dropped the window size to 8k (iperf -w8k) the 
> throughput
> > is 750mbs.
> >
> > Any suggestions, ideas?
>
>Some troubleshooting tips:
>
>Are some packets lost on the router? Checking packet counters
>might give you a clue.
>
>Do you see some errors on one of the IB nodes?
>Set debug_level=1 module parameter for ib_ipoib, and check
>dmesg output while running the test.
>
>--
>MST