[openib-general] ip over ib throughtput

[Shirley Ma]

I am testing most of my patches. Under

1.Intel(R) Xeon(TM) CPU 2.80GHz, one cpu, 
2. fw-23108-3_4_000-MHXL-CF128-T.bin
3. pci-x without msi_x enabled
4. kernel 2.6.16
5. netperf-2.4.0
6. SVN 68XX+several IPoIB patches

The best result I got so far:

Testing with the following command line:
netperf -l 60 -H 10.1.1.100 -t TCP_STREAM -i 10,2 -I 95,5 -- -m 16384 -s 
349520 -S 349520

TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.1.1.100 
(10.1.1.100) port 0 AF_INET : +/-2.5% @ 95% conf.
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

699040 699040  16384    60.00    3668.07 (458MB/s)

cpu utilization was around 95%.

Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638





"Hassan M. Jafri" <hjafri at ncsa.uiuc.edu> 
Sent by: openib-general-bounces at openib.org
05/08/2006 03:52 PM

To
openib-general at openib.org
cc

Subject
Re: [openib-general] ip over ib throughtput






I cant crank out more than 150 MB/sec with my 2.0 GHz xeons. verbs level 
benchmarks, however give decent numbers for bandwidth. With netperf, the 
server side CPU usage is 99% which is much higher than other posted 
bandwidth results on this thread. Any suggestions?

Here is the complete configuration for my bandwidth tests

Kernel-2.6.15.4
netperf-2.3-3
OpenIB rev 6552
MTLP23108-CF128
Firmware 3.4.0
MSI-X is enabled for the HCA


------------------------------
Here is the netperf output



TCP STREAM TEST to 192.168.2.2
Recv   Send    Send                          Utilization       Service 
Demand
Socket Socket  Message  Elapsed              Send     Recv     Send Recv
Size   Size    Size     Time     Throughput  local    remote   local 
remote
bytes  bytes   bytes    secs.    MBytes  /s  % T      % T      us/KB us/KB

262142 262142  32768    10.01       151.32   59.66    99.84    7.700 
12.886
-------------------------------

Here is ib0 config for one of the nodes

ib0       Link encap:UNSPEC  HWaddr 
00-02-04-04-FE-80-00-00-00-00-00-00-00-00-00-00
           inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
           inet6 addr: fe80::202:c902:0:3ce9/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
           RX packets:1724527 errors:0 dropped:0 overruns:0 frame:0
           TX packets:9685456 errors:0 dropped:2 overruns:0 carrier:0
           collisions:0 txqueuelen:128
           RX bytes:89830114 (85.6 MiB)  TX bytes:2213308646 (2.0 GiB)








Michael S. Tsirkin wrote:
> Hi!
> What kind of performance do people see with ip over ib on gen2?
> I see about 100Mbyte/sec at 99% CPU utilisation on send,
> on an express card, Xeon 2.8GHz, SSE doorbells enabled.
> 
> MST
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
> 
_______________________________________________
openib-general mailing list
openib-general at openib.org
http://openib.org/mailman/listinfo/openib-general

To unsubscribe, please visit 
http://openib.org/mailman/listinfo/openib-general

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060508/ae0cf277/attachment.html>