<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=Big5-HKSCS" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hal Rosenstock wrote:<br>
<blockquote cite="mid1149520880.4510.210194.camel@hal.voltaire.com"
type="cite">
<pre wrap="">On Mon, 2006-06-05 at 11:12, hbchen wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
I have a question about the IPoIB bandwidth performance.
I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G
ethernet card,
and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface).
NIC (Jumbo enabled) Line bandwidth(LB) IPoverNIC bandwidth utilization
(IPoNIC/LB)
--------------------- ---------------- --------------
----------------------------------
Single Gigabit NIC : 1Gb/sec=125MB/sec 120MB/sec 96% (PIC-X interface)
Myrinet D card : 250MB/sec 240~-245MB/sec 96% ~ 98% (PCI-X interface)
Myrinet 10G Ethernet: 10Gb/sec=1280MB/sec 980MB/sec 76.6% (My testing
using Linux 2.6.14.6)
(PCI-Express) 1225MB/sec 95.7% (Data from Myrinet website)
IB HCA4X(PCI-Express): 10Gb/sec=1280MB/sec 420MB/sec 32.8% (My testing
using Linux 2.6.14.6)
474MB/sec 37% (the best from OpenIB mailing list)
(2.6.12-rc5 patch 1)
Why the bandwidth utilization of IPoIB is so low compared to the others
NICs?
</pre>
</blockquote>
<pre wrap=""><!---->
One thing to note is that the max utilization of 10G IB (4x) is 8G due
to the signalling being included in this rate (unlike ethernet whose
rate represents the data rate and does not include the signalling
overhead).
</pre>
</blockquote>
Hal,<br>
Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth
utilization is still very low.<br>
>> IPoIB=420MB/sec <br>
>> bandwidth utilization= 420/1024 = 41.01%<br>
<br>
<br>
HB <br>
<br>
<br>
<br>
<blockquote cite="mid1149520880.4510.210194.camel@hal.voltaire.com"
type="cite">
<pre wrap="">
-- Hal
</pre>
<blockquote type="cite">
<pre wrap="">There must be a lot of room to improve the IPoIB software to reach 75%+
bandwidth utilization.
HB Chen
Los Alamos National Lab
<a class="moz-txt-link-abbreviated" href="mailto:hbchen@labl.gov">hbchen@labl.gov</a>
_______________________________________________
openib-general mailing list
<a class="moz-txt-link-abbreviated" href="mailto:openib-general@openib.org">openib-general@openib.org</a>
<a class="moz-txt-link-freetext" href="http://openib.org/mailman/listinfo/openib-general">http://openib.org/mailman/listinfo/openib-general</a>
To unsubscribe, please visit <a class="moz-txt-link-freetext" href="http://openib.org/mailman/listinfo/openib-general">http://openib.org/mailman/listinfo/openib-general</a>
</pre>
</blockquote>
<pre wrap=""><!---->
</pre>
</blockquote>
<br>
</body>
</html>