[ewg] IPoIB to Ethernet routing performance

[sebastien dugue]

  Hi Jabe,

On Mon, 06 Dec 2010 21:47:42 +0100
Jabe <jabe.chapman at shiftmail.org> wrote:

> 
> >    The router is fitted with one ConnectX2 QDR HCA and one dual port Myricom 10G
> > Ethernet adapter.
> >
> > ...
> >
> >    Here are some numbers:
> >
> >    - 1 IPoIB stream between client and router: 20 Gbits/sec
> >
> >      Looks OK.
> >
> >    - 2 Ethernet streams between router and server: 19.5 Gbits/sec
> >
> >      Looks OK.
> >    
> 
> 
> Actually I am amazed you can get such a speed with IPoIB. Trying with 
> NPtcp on my DDR infiniband I can only obtain about 4.6Gbit/sec at the 
> best packet size (that is 1/4 of the infiniband bandwidth) with this 
> chip embedded in the mainboard: InfiniBand: Mellanox Technologies 
> MT25204 [InfiniHost III Lx HCA]; and dual E5430 xeon (not nehalem).
> That's with 2.6.37 kernel and vanilla ib_ipoib module. What's wrong with 
> my setup?
> I always assumed that such a slow speed was due to the lack of 
> offloading capabilities you get with ethernet cards, but maybe I was 
> wrong...?
> Also what application did you use for the benchmark?

 I'm using iperf.

  Sébastien.

> Thank you
>