[ofa-general] Low NFS RDMA performance with Connect X

[James Lentini]

On Wed, 5 Sep 2007, Kuchimanchi, Ramachandra wrote:

> John Leidel <john.leidel at gmail.com> wrote:
>  
> > In doing some testing with ConnectX, I noticed a similar issue in MPI
> > performance.  The fix was simply to upgrade to the latetest and greatest
> > firmware.
> 
> I tried with the latest ConnectX Firmware, version 2.2, and the Iozone
> numbers are almost similar to what I posted previously and very low as
> compared to the MT25208 numbers.
> 
> NFS RDMA folks, any ideas as to why this is happening with Connect X ?

We are bringing up our Connect X systems now (we're waiting on a 
replacement memory dimm for our server). We'll be experimenting with 
the performance on Connect X over the next few weeks.

Both the client and server code bases have been updated substantially 
since the Mellanox SDK was released. Results are likely to be 
different on the newer code.

Finally, it is conceivable that there will need to be performance 
tweeks for the Connect X hardware. For Tavor hardware, ULPs use a 1KB 
MTU to achieve maximum performance (see the setup of the path_mtu QP 
attribute in net/sunrpc/xprtrdma/verbs.c).