<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7651.59">
<TITLE>Low NFS RDMA performance with Connect X</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->
<P><FONT SIZE=2>Hi,<BR>
<BR>
I took the NFS RDMA code from the Mellanox NFS RDMA SDK, compiled it with<BR>
OFED-1.2.5 and tried it out with Connect X HCAs and also MT25208. I found<BR>
that the Iozone read and write performance numbers are very low on Connect X.<BR>
<BR>
For a 128 MB file and a 128 KB record size<BR>
<BR>
NFS RDMA SDK on MT25028: Read: 861 MB/s Write: 185 MB/s<BR>
OFED-1.2.5 with NFS RDMA modules Read: 849 MB/s Write: 184 MB/s <BR>
on a MT25208<BR>
OFED-1.2.5 with NFS RDMA modules Read: 451 MB/s Write: 79 MB/s<BR>
on Connect X<BR>
<BR>
Has any one tried this out or know of a reason why the numbers are so low<BR>
on Connect X ?<BR>
<BR>
Test-setup:<BR>
Server and single client running RHEL 5<BR>
MT25208 tests were with dual processor 64-bit AMD machines<BR>
Connect X tests were with dual processor dual core 64-bit AMD machines<BR>
Connect X HCA FW ver: 2.1<BR>
NFS mount was in async mode and iozone tests were run with -c option.<BR>
<BR>
More Iozone results for a record size of 64 KB (values below in KB/sec):<BR>
<BR>
Read test <BR>
<BR>
File Size SDK on MT25208 OFED-1.2.5 on MT25208 OFED-1.2.5 on ConnectX<BR>
(in MB) <BR>
64 1684819 1701916 459279<BR>
128 882580 870180 462486<BR>
256 922081 921932 468063<BR>
512 871136 909221 452969<BR>
1024 900314 910171 442215<BR>
2048 908117 849710 676776<BR>
<BR>
Write test <BR>
<BR>
File Size SDK on MT25208 OFED-1.2.5 on MT25208 OFED-1.2.5 on ConnectX<BR>
(in MB)<BR>
64 184154 182483 78424<BR>
128 190126 189284 81869<BR>
256 194921 173124 85813<BR>
512 199666 192110 87628<BR>
1024 208924 199240 126415<BR>
2048 180128 195278 123020<BR>
<BR>
Regards,<BR>
Ram<BR>
</FONT>
</P>
</BODY>
</HTML>