[ofa-general] Low NFS RDMA performance with Connect X

[Kuchimanchi, Ramachandra]

John Leidel <john.leidel at gmail.com> wrote:
 
> In doing some testing with ConnectX, I noticed a similar issue in MPI
> performance.  The fix was simply to upgrade to the latetest and greatest
> firmware.

I tried with the latest ConnectX Firmware, version 2.2, and the Iozone
numbers are almost similar to what I posted previously and very low as
compared to the MT25208 numbers.

NFS RDMA folks, any ideas as to why this is happening with Connect X ?

Regards,
Ram

On 9/4/07, Kuchimanchi, Ramachandra <ramachandra.kuchimanchi at qlogic.com>
wrote:
>
>  Hi,
>
> I took the NFS RDMA code from the Mellanox NFS RDMA SDK, compiled it with
> OFED-1.2.5 and tried it out with Connect X HCAs and also MT25208. I found
> that the Iozone read and write performance numbers are very low on Connect
> X.
>
> For a 128 MB file and a 128 KB record size
>
> NFS RDMA SDK on MT25028:              Read: 861 MB/s       Write: 185 MB/s
> OFED-1.2.5 with NFS RDMA modules      Read: 849 MB/s       Write: 184
> MB/s
> on a MT25208
> OFED-1.2.5 with NFS RDMA modules      Read: 451 MB/s       Write: 79 MB/s
> on Connect X
>
> Has any one tried this out or know of a reason why the numbers are so low
> on Connect X ?
>
> Test-setup:
> Server and single client running RHEL 5
> MT25208 tests were with dual processor 64-bit AMD machines
> Connect X tests were with dual processor dual core 64-bit AMD machines
> Connect X HCA FW ver: 2.1
> NFS mount was in async mode and iozone tests were run with -c option.
>
> More Iozone results for a record size of 64 KB (values below in KB/sec):
>
> Read test
>
> File Size  SDK on MT25208      OFED-1.2.5 on MT25208   OFED-1.2.5 on
> ConnectX
> (in MB)
> 64           1684819               1701916                   459279
> 128          882580                 870180                   462486
> 256          922081                 921932                   468063
> 512          871136                 909221                   452969
> 1024         900314                 910171                   442215
> 2048         908117                 849710                   676776
>
> Write test
>
> File Size  SDK on MT25208    OFED-1.2.5 on MT25208   OFED-1.2.5 on
> ConnectX
> (in MB)
> 64           184154                 182483                   78424
> 128          190126                 189284                   81869
> 256          194921                 173124                   85813
> 512          199666                 192110                   87628
> 1024         208924                 199240                   126415
> 2048         180128                 195278                   123020
>
> Regards,
> Ram
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070905/2d4c5d81/attachment.html>