[openib-general] Question about the IPoIB bandwidth performance ?

[Felix Marti]

________________________________

From: openib-general-bounces at openib.org
[mailto:openib-general-bounces at openib.org] On Behalf Of hbchen
Sent: Monday, June 05, 2006 9:12 AM
To: Talpey, Thomas
Cc: openib-general at openib.org
Subject: Re: [openib-general] Question about the IPoIB bandwidth
performance ?

 

Talpey, Thomas wrote:



At 11:38 AM 6/5/2006, hbchen wrote:
  

	Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth
utilization is still very low.
	    

			IPoIB=420MB/sec  
			bandwidth utilization= 420/1024 = 41.01%
			        

 
 
Helen, have you measured the CPU utilizations during these runs?
Perhaps you are out of CPU.
 
  

Tom,
I am HB Chen from LANL not the Helen Chen from SNL.
I didn't run out of CPU.  It is about 70-80 % of CPU utilization.
  



Outrageous opinion follows.
 
Frankly, an IB HCA running Ethernet emulation is approximately the
world's worst 10GbE adapter (not to put too fine of a point on it :-) )
  

The IP over Myrinet ( Ethernet emulation) can reach upto 96%-98%
bandwidth utilization why not the IPoIB ?



[Felix:] As pointed out earlier: it is the message rate. If you change
the mtu to 1500B (instead of the non-standard 9000B Jumbo frames)
performance will drop into the same range as what you see with IPoIB
(limited by the receiver).


HB Chen 
hbchen at lanl.gov



There is no hardware checksumming, nor large-send offloading, both
of which force overhead onto software. And, as you just discovered
it isn't even 10Gb!
 
In general, network emulation layers are always going to perform more
poorly than native implementations. But this is only a generality
learned
from years of experience with them.
 
Tom.  
 
  

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060605/e152ba96/attachment.html>