[openib-general] ip over ib throughtput

[Josh England]

I'm seeing about 364 MB/s between 2 PCIe Xeon 3.2GHz boxes using
netperf-2.3pl1.

-JE

On Tue, 2005-01-04 at 19:47 +0000, Paul Baxter wrote:
> > Quoting r. Michael S. Tsirkin (mst at mellanox.co.il) "[openib-general] ip 
> > over ib throughtput":
> >> Hi!
> >> What kind of performance do people see with ip over ib on gen2?
> >> I see about 100Mbyte/sec at 99% CPU utilisation on send,
> >> on an express card, Xeon 2.8GHz, SSE doorbells enabled.
> >
> > At 159 Mbyte/sec without SSE doorbells.
> > The test is netperf-2.2pl5, BTW.
> 
> Didn't see any response to Michael's numbers other than another mail today 
> from Josh England saying 'great numbers till it dies'.
> 
> Are these results considered good on a 4x link? I realise there is a 
> significant TCPIP overhead, but is this significantly better or worse than 
> the vendor-specific equivalents using different drivers?
> 
> What (short term?) plans are there for implementing a higher performance 
> link that can show better transfer rates.
> 
> I'm interested in any bottlenecks this might reveal in the current kernel 
> submission and understanding in broad handfuls what sort of 
> optimisation/stabilisation period will be necessary before I can look to 
> using openib in high bandwidth message transfers. Is more functionality or 
> performance optimisation the goal for the next 6 months.
> 
> I've largely written off SDP going higher than ~300MB/s even with high CPU 
> utilisation. (Assumptions: License concerns make it unlikely to be one of 
> the first things openib tackle. Not trivial to implement it with zero copy 
> or asynchronous I/O on Linux)
> 
> Am I right in thinking that the ib_verbs layer direct or ideally with 
> MPI/uDAPL will be my best bet in the next 6 months for showing a portable 
> vendor-neutral implementation which might achieve 600MB/s transfers or 
> slightly lower but with <25% cpu utilisation on PCIe.
> 
> Paul 
> 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>