<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7651.59">
<TITLE>RE: [ofa-general] Low NFS RDMA performance with Connect X</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>John Leidel <john.leidel@gmail.com> wrote:<BR>
<BR>
> In doing some testing with ConnectX, I noticed a similar issue in MPI<BR>
> performance. The fix was simply to upgrade to the latetest and greatest<BR>
> firmware.<BR>
<BR>
I tried with the latest ConnectX Firmware, version 2.2, and the Iozone<BR>
numbers are almost similar to what I posted previously and very low as<BR>
compared to the MT25208 numbers.<BR>
<BR>
NFS RDMA folks, any ideas as to why this is happening with Connect X ?<BR>
<BR>
Regards,<BR>
Ram<BR>
<BR>
On 9/4/07, Kuchimanchi, Ramachandra <ramachandra.kuchimanchi@qlogic.com><BR>
wrote:<BR>
><BR>
> Hi,<BR>
><BR>
> I took the NFS RDMA code from the Mellanox NFS RDMA SDK, compiled it with<BR>
> OFED-1.2.5 and tried it out with Connect X HCAs and also MT25208. I found<BR>
> that the Iozone read and write performance numbers are very low on Connect<BR>
> X.<BR>
><BR>
> For a 128 MB file and a 128 KB record size<BR>
><BR>
> NFS RDMA SDK on MT25028: Read: 861 MB/s Write: 185 MB/s<BR>
> OFED-1.2.5 with NFS RDMA modules Read: 849 MB/s Write: 184<BR>
> MB/s<BR>
> on a MT25208<BR>
> OFED-1.2.5 with NFS RDMA modules Read: 451 MB/s Write: 79 MB/s<BR>
> on Connect X<BR>
><BR>
> Has any one tried this out or know of a reason why the numbers are so low<BR>
> on Connect X ?<BR>
><BR>
> Test-setup:<BR>
> Server and single client running RHEL 5<BR>
> MT25208 tests were with dual processor 64-bit AMD machines<BR>
> Connect X tests were with dual processor dual core 64-bit AMD machines<BR>
> Connect X HCA FW ver: 2.1<BR>
> NFS mount was in async mode and iozone tests were run with -c option.<BR>
><BR>
> More Iozone results for a record size of 64 KB (values below in KB/sec):<BR>
><BR>
> Read test<BR>
><BR>
> File Size SDK on MT25208 OFED-1.2.5 on MT25208 OFED-1.2.5 on<BR>
> ConnectX<BR>
> (in MB)<BR>
> 64 1684819 1701916 459279<BR>
> 128 882580 870180 462486<BR>
> 256 922081 921932 468063<BR>
> 512 871136 909221 452969<BR>
> 1024 900314 910171 442215<BR>
> 2048 908117 849710 676776<BR>
><BR>
> Write test<BR>
><BR>
> File Size SDK on MT25208 OFED-1.2.5 on MT25208 OFED-1.2.5 on<BR>
> ConnectX<BR>
> (in MB)<BR>
> 64 184154 182483 78424<BR>
> 128 190126 189284 81869<BR>
> 256 194921 173124 85813<BR>
> 512 199666 192110 87628<BR>
> 1024 208924 199240 126415<BR>
> 2048 180128 195278 123020<BR>
><BR>
> Regards,<BR>
> Ram<BR>
><BR>
> _______________________________________________<BR>
> general mailing list<BR>
> general@lists.openfabrics.org<BR>
> <A HREF="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</A><BR>
><BR>
> To unsubscribe, please visit<BR>
> <A HREF="http://openib.org/mailman/listinfo/openib-general">http://openib.org/mailman/listinfo/openib-general</A><BR>
><BR>
<BR>
</FONT>
</P>
</BODY>
</HTML>