<html>
<body>
<font size=3>At 08:20 AM 10/19/2007, Peter Kjellstrom wrote:<br>
<blockquote type=cite class=cite cite="">On Thursday 18 October 2007,
Chuck Hartley wrote:<br>
...<br>
> 8388608
5000
1342.12
1342.12<br>
>
------------------------------------------------------------------<br>
><br>
> Is this typical RDMA performance?<br><br>
It's close to what I've seen on similar hw. ~1400 is what you can push
through <br>
the 8x pci-e of the intel 5000 chipset (confirmed by trying 4x pci-e
which <br>
has shown ~700).<br><br>
> What is the maximum theoretical BW for <br>
> DDR IB - 1525MB/sec?<br><br>
No, it's 20 Gbps on the wire and 8/10 encoded so 16 Gbps effective which
is <br>
2000 MB/s (10-base) and 1907 MiB/s (2-base).</font></blockquote><br>
There is also IB protocol overhead combined with driver / device control
traffic overhead (consumes device as well as PCI resources / bandwidth),
end-to-end control traffic which is also a function of how the
application is constructed. In general, hitting about 80-85%
of the theoretical maximum is possible.<br><br>
<blockquote type=cite class=cite cite=""><font size=3>On our system (with
a different HCA) we see quite a difference with <br>
snoop-filter off (bios option). With snoop off (our) application
performance <br>
goes up (not very suprising) but IB performance goes down (latency 0.4us
<br>
worse and bw ~1400->1200).</blockquote><br>
Mike</font></body>
</html>