[openib-general] Question about the IPoIB bandwidth performance ?

[hbchen]

Hal Rosenstock wrote:

>On Mon, 2006-06-05 at 11:12, hbchen wrote:
>  
>
>>Hi,
>>I have a question about the IPoIB bandwidth performance.
>>I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G
>>ethernet card,
>>and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface).
>>
>>
>>NIC (Jumbo enabled) Line bandwidth(LB) IPoverNIC bandwidth utilization
>>(IPoNIC/LB)
>>--------------------- ---------------- --------------
>>----------------------------------
>>Single Gigabit NIC : 1Gb/sec=125MB/sec 120MB/sec 96% (PIC-X interface)
>>Myrinet D card : 250MB/sec 240~-245MB/sec 96% ~ 98% (PCI-X interface)
>>Myrinet 10G Ethernet: 10Gb/sec=1280MB/sec 980MB/sec 76.6% (My testing
>>using Linux 2.6.14.6)
>>(PCI-Express) 1225MB/sec 95.7% (Data from Myrinet website)
>>IB HCA4X(PCI-Express): 10Gb/sec=1280MB/sec 420MB/sec 32.8% (My testing
>>using Linux 2.6.14.6)
>>474MB/sec 37% (the best from OpenIB mailing list)
>>(2.6.12-rc5 patch 1)
>>
>>Why the bandwidth utilization of IPoIB is so low compared to the others
>>NICs?
>>    
>>
>
>One thing to note is that the max utilization of 10G IB (4x) is 8G due
>to the signalling being included in this rate (unlike ethernet whose
>rate represents the data rate and does not include the signalling
>overhead).
>  
>
Hal,
Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth
utilization is still very low.
>> IPoIB=420MB/sec
>> bandwidth utilization= 420/1024 = 41.01%


HB



>-- Hal
>
>  
>
>>There must be a lot of room to improve the IPoIB software to reach 75%+
>>bandwidth utilization.
>>
>>
>>HB Chen
>>Los Alamos National Lab
>>hbchen at labl.gov
>>
>>_______________________________________________
>>openib-general mailing list
>>openib-general at openib.org
>>http://openib.org/mailman/listinfo/openib-general
>>
>>To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>>
>>    
>>
>
>  
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060605/bff889b1/attachment.html>