In doing some testing with ConnectX, I noticed a similar issue in MPI performance. The fix was simply to upgrade to the latetest and greatest firmware. <br><br><div><span class="gmail_quote">On 9/4/07, <b class="gmail_sendername">
Kuchimanchi, Ramachandra</b> <<a href="mailto:ramachandra.kuchimanchi@qlogic.com">ramachandra.kuchimanchi@qlogic.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<p><font size="2">Hi,<br>
<br>
I took the NFS RDMA code from the Mellanox NFS RDMA SDK, compiled it with<br>
OFED-1.2.5 and tried it out with Connect X HCAs and also MT25208. I found<br>
that the Iozone read and write performance numbers are very low on Connect X.<br>
<br>
For a 128 MB file and a 128 KB record size<br>
<br>
NFS RDMA SDK on MT25028: Read: 861 MB/s Write: 185 MB/s<br>
OFED-1.2.5 with NFS RDMA modules Read: 849 MB/s Write: 184 MB/s <br>
on a MT25208<br>
OFED-1.2.5 with NFS RDMA modules Read: 451 MB/s Write: 79 MB/s<br>
on Connect X<br>
<br>
Has any one tried this out or know of a reason why the numbers are so low<br>
on Connect X ?<br>
<br>
Test-setup:<br>
Server and single client running RHEL 5<br>
MT25208 tests were with dual processor 64-bit AMD machines<br>
Connect X tests were with dual processor dual core 64-bit AMD machines<br>
Connect X HCA FW ver: 2.1<br>
NFS mount was in async mode and iozone tests were run with -c option.<br>
<br>
More Iozone results for a record size of 64 KB (values below in KB/sec):<br>
<br>
Read test <br>
<br>
File Size SDK on MT25208 OFED-1.2.5 on MT25208 OFED-1.2.5 on ConnectX<br>
(in MB) <br>
64 1684819 1701916 459279<br>
128 882580 870180 462486<br>
256 922081 921932 468063<br>
512 871136 909221 452969<br>
1024 900314 910171 442215<br>
2048 908117 849710 676776<br>
<br>
Write test <br>
<br>
File Size SDK on MT25208 OFED-1.2.5 on MT25208 OFED-1.2.5 on ConnectX<br>
(in MB)<br>
64 184154 182483 78424<br>
128 190126 189284 81869<br>
256 194921 173124 85813<br>
512 199666 192110 87628<br>
1024 208924 199240 126415<br>
2048 180128 195278 123020<br>
<br>
Regards,<br>
Ram<br>
</font>
</p>
</div>
<br>_______________________________________________<br>general mailing list<br><a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:general@lists.openfabrics.org">general@lists.openfabrics.org</a><br><a onclick="return top.js.OpenExtLink(window,event,this)" href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" target="_blank">
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br><br>To unsubscribe, please visit <a onclick="return top.js.OpenExtLink(window,event,this)" href="http://openib.org/mailman/listinfo/openib-general" target="_blank">
http://openib.org/mailman/listinfo/openib-general</a><br></blockquote></div><br>