[ofa-general][PATCH] NFS-RDMA fix for connectX fast registration wr,

[Vu Pham]

Tom,

> Vu - I noticed this wasn't a usual kernel patch, so I've recoded it
> as a patch against Trond's tree, where I have the connectathon
> patches queued too. I can send it to him with your signed-off-by,
> if that's ok.
>
>   
Here is the kernel patch. It's tested with latest connectX FW (2.6.636)


> I do want to be sure the patch is tested though. Does it work on
> earlier COnnectX firmware that may not support FRMR? I assume
> that fw doesn't set the bits the verbs.c file is looking for to detect
> FRMR support... correct?
>   

The patch does not work with old FW.
The old fw set the bit; however, it has bug on FRWR

thanks,
-vu

> Tom.
>
> At 06:09 PM 3/22/2009, Tom Talpey wrote:
>   
>> At 02:27 PM 3/22/2009, James Lentini wrote:
>>     
>>> On Fri, 20 Mar 2009, Vu Pham wrote:
>>>
>>>       
>>>> Hi Jon,
>>>>
>>>> Here is the patch for the file
>>>> kernel_patches/fixes/nfsrdma_to_2_6_28_18.patch. This patch will 
>>>>         
>> fix the fast
>>     
>>>> registration wr for Mellanox ConnectX.
>>>>
>>>> ConnectX FRMR requires local write enable together with remote rdma write
>>>> enable
>>>>
>>>> thanks,
>>>> -vu
>>>>
>>>>         
>>> Vu,
>>>
>>> Are you going to submit this fix upstream?
>>>       
>> Agreed - if setting this bit fixes the FRMR issue, then it should go to
>> Trond asap for inclusion in 2.6.30 (hopefully). Can someone verify that
>> it works? I don't have access to ConnectX hardware as I am at the
>> OFA workshop.
>>
>> Tom.
>>
>>     
>>> -james
>>>
>>>
>>>       
>>>> --- 
>>>>         
>>> ofa_kernel-1.4.1.config/kernel_patches/fixes/nfsrdma_to_2.6.28_13.patc
>>> h	2009-03-19 08:42:07.000000000 -0700
>>>       
>>>> +++ ofa_kernel-1.4.1/kernel_patches/fixes/nfsrdma_to_2.6.28_13.patch	
>>>>         
>>> 2009-03-19 08:49:22.000000000 -0700
>>>       
>>>> @@ -167,7 +167,7 @@
>>>>   	case RPCRDMA_MTHCAFMR:
>>>>   	case RPCRDMA_MEMWINDOWS_ASYNC:
>>>>   	case RPCRDMA_MEMWINDOWS:
>>>> -@@ -1391,6 +1457,96 @@ rpcrdma_unmap_one(struct rpcrdma_ia *ia, 
>>>>         
>>> struct rpcrdma_mr_seg *seg)
>>>       
>>>> +@@ -1391,6 +1457,98 @@ rpcrdma_unmap_one(struct rpcrdma_ia *ia, 
>>>>         
>>> struct rpcrdma_mr_seg *seg)
>>>       
>>>>   }
>>>>   
>>>>   static int
>>>> @@ -214,8 +214,10 @@
>>>>  +	frmr_wr.wr.fast_reg.page_list_len = i;
>>>>  +	frmr_wr.wr.fast_reg.page_shift = PAGE_SHIFT;
>>>>  +	frmr_wr.wr.fast_reg.length = i << PAGE_SHIFT;
>>>> -+	frmr_wr.wr.fast_reg.access_flags = (writing ?
>>>> -+				IB_ACCESS_REMOTE_WRITE : IB_ACCESS_REMOTE_READ);
>>>> ++	frmr_wr.wr.fast_reg.access_flags =
>>>> ++		(writing ?
>>>> ++		(IB_ACCESS_REMOTE_WRITE | IB_ACCESS_LOCAL_WRITE) :
>>>> ++		IB_ACCESS_REMOTE_READ);
>>>>  +	frmr_wr.wr.fast_reg.rkey = seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey;
>>>>  +	DECR_CQCOUNT(&r_xprt->rx_ep);
>>>>  +
>>>>         
>
>   

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: enable_local_write_connectX.patch
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20090323/b65b1fd0/attachment.ksh>