<html>
<body>
<font size=3>At 02:33 PM 10/20/2007, Peter Kjellstrom wrote:<br>
<blockquote type=cite class=cite cite="">On Friday 19 October 2007,
Michael Krause wrote:<br>
> At 08:20 AM 10/19/2007, Peter Kjellstrom wrote:<br>
> >On Thursday 18 October 2007, Chuck Hartley wrote:<br>
...<br>
> > > What is the maximum theoretical BW for<br>
> > > DDR IB - 1525MB/sec?<br>
> ><br>
> >No, it's 20 Gbps on the wire and 8/10 encoded so 16 Gbps
effective which<br>
> > is 2000 MB/s (10-base) and 1907 MiB/s (2-base).<br>
><br>
> There is also IB protocol overhead combined with driver / device
control<br>
> traffic overhead (consumes device as well as PCI resources /
bandwidth),<br>
> end-to-end control traffic which is also a function of how the
application<br>
> is constructed. In general, hitting about 80-85% of the
theoretical<br>
> maximum is possible.<br><br>
IB can do much better than that. On an SDR system I typically get 950
MB/s <br>
(10-base), 95%. This on 8x pci-express so the limitations of pci-e above
does <br>
not bite. If IB DDR could strech it's legs (if we had faster pci-e, say
<br>
pci-e-2.0...) then maybe we would see 95% there too
:-).</blockquote><br>
While there are certainly marketing workloads that can hit such high
efficiencies, the number of real world workloads is rather
small. There was one interconnect provider a few years back
who used to demonstrate 95+% efficiency post 8b/10b encoding overhead by
sending 1MB messages so the host interaction was to pull a single work
request and then just issue DMA Read Requests. Just like that they
were at link rate. However, most workloads are not single streams
but a mix of streams with varying work request rates, sizes, etc. I
don't doubt that one can hit higher rates than 80-85% but expect most
workloads to rarely exceed this value. A couple years ago a
reporter asked me about why some interconnects are at 50-60% efficiency
when measured in real environments. We walked through the host /
device as well as driver interactions, the ability of the platform to
actually generate useful I/O work (some are processor / memory limited so
improvements in the I/O subsystems has no real ROI), the protocol
overheads, etc. He was trying ascertain whether there was a story
here about vendors basically hyping their technology using the various
marketing numbers when in reality they could not actually deliver the
performance under more than a contrived or limited set of
workloads. I convinced him there was no story here but it did
illustrate my earlier industry talks about marketing hype vs. reality and
how marketing does a great deal of harm due to lost credibility when it
comes to running real world applications.<br><br>
Mike</font></body>
</html>