[ofa-general] Re: [PATCH] NFS-RDMA fix for connectX fast registration wr,

[Jon Mason]

On Fri, Mar 20, 2009 at 04:13:00PM -0700, Vu Pham wrote:
> Hi Jon,
>
> Here is the patch for the file  
> kernel_patches/fixes/nfsrdma_to_2_6_28_18.patch. This patch will fix the  
> fast registration wr for Mellanox ConnectX.

I was able to run Connectathon over Chelsio adapters without issue.
Assuming everyone agrees on this solution, this patch should be safe for
inclusion into the backports.

Thanks,
Jon

>
> ConnectX FRMR requires local write enable together with remote rdma  
> write enable
>
> thanks,
> -vu

> --- ofa_kernel-1.4.1.config/kernel_patches/fixes/nfsrdma_to_2.6.28_13.patch	2009-03-19 08:42:07.000000000 -0700
> +++ ofa_kernel-1.4.1/kernel_patches/fixes/nfsrdma_to_2.6.28_13.patch	2009-03-19 08:49:22.000000000 -0700
> @@ -167,7 +167,7 @@
>   	case RPCRDMA_MTHCAFMR:
>   	case RPCRDMA_MEMWINDOWS_ASYNC:
>   	case RPCRDMA_MEMWINDOWS:
> -@@ -1391,6 +1457,96 @@ rpcrdma_unmap_one(struct rpcrdma_ia *ia, struct rpcrdma_mr_seg *seg)
> +@@ -1391,6 +1457,98 @@ rpcrdma_unmap_one(struct rpcrdma_ia *ia, struct rpcrdma_mr_seg *seg)
>   }
>   
>   static int
> @@ -214,8 +214,10 @@
>  +	frmr_wr.wr.fast_reg.page_list_len = i;
>  +	frmr_wr.wr.fast_reg.page_shift = PAGE_SHIFT;
>  +	frmr_wr.wr.fast_reg.length = i << PAGE_SHIFT;
> -+	frmr_wr.wr.fast_reg.access_flags = (writing ?
> -+				IB_ACCESS_REMOTE_WRITE : IB_ACCESS_REMOTE_READ);
> ++	frmr_wr.wr.fast_reg.access_flags =
> ++		(writing ?
> ++		 (IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE) :
> ++		 IB_ACCESS_REMOTE_READ);
>  +	frmr_wr.wr.fast_reg.rkey = seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey;
>  +	DECR_CQCOUNT(&r_xprt->rx_ep);
>  +