[ewg] Re: [PATCH] NFS-RDMA backport for RHEL 5.2

[Jon Mason]

On Mon, Mar 02, 2009 at 05:37:45PM +0200, Vladimir Sokolovsky wrote:
> Jon Mason wrote:
>> Hey Vlad,
>>
>> I wanted to get you the NFS-RDMA backport patches I have queued up prior
>> to rc1 being built.  I have this patch (RHEL5.2), as well as 2.6.22, and
>> 2.6.25 (which I will be sending in separate e-mails).
>>
>> This patch contains the changes necessary to get NFS-RDMA 99% working on
>> RHEL5.2.  It 100% passes of the connectathon testcases when using RDMA.
>> When using TCP, 1 testcase hangs (for which I've found a code hack to
>> work around, but not a fix).  The testcase is the 30MB transfer in the
>> "special" tests, and is 100% reproducible.
>>
>> With the attached patch, you can now enable the nfs-rdma-mod by default
>> and run it through your nightly build tests.
>>
>> There is one problem with nfs-utils, which will necessitate including it
>> in the OFED distribution.  Currently, there is a check for the kernel
>> version you are running when running the mount.nfs command.  This check
>> is to verify if one is running a kernel greater than 2.6.22.  If not,
>> then it will not allow a mount with RDMA as an option.  In earlier
>> versions, there was an override option of "-i".  This has since been
>> removed for an unknown reason (but if you run `mount.nfs`, you can see
>> that they forgot to remove 'i' from "usage" options).  I will send them
>> a patch to fix this in the latest version, but this will be a problem
>> that will require it to be built locally until such a time as that is
>> pulled in.
>>
>> In the OFED tree, there is another problem that needs to be addressed.
>> There are kernel header files that are being automatically imported via
>> the OFED tree.  The redefinition of these kernel data structs causes
>> major problems.  Specifically, these header files are
>> include/linux/pipe_fs_i.h and include/linux/splice.h.  These files
>> redefine pipe_inode_info and pipe_buf_operations and many other structs
>> used in OFED and the underlying kernel.  They should NOT be included,
>> and can cause problems when backporting NFS-RDMA to other kernel
>> versions.  I have removed their usage in my patches, but they should be
>> removed from the OFED tree as they should not be needed (as we want to
>> use the native header files for the kernel we are compiling against).
>>
>> Thanks,
>> Jon
>>   
> Hi Jon,
> Is there any reason that you removed sg_next function from  
> kernel_addons/backport/2.6.18-EL5.2/include/linux/scatterlist.h?
>
> -static inline struct scatterlist *sg_next(struct scatterlist *sg)
> -{
> -       if (!sg) {
> -               BUG();
> -               return NULL;
> -       }
> -       return sg + 1;
> -}
> -

Oops, I must've removed it on accident.  NFS-RDMA has no need for it not
to be there.

If you want, I can provide a patch (or a new patchset for NFS-RDMA) to
fix this issue.

Thanks,
Jon

>
> This cause iSER compilation failure on RHEL5.2:
>
> /home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.c:253: warning: assignment makes pointer from integer without a cast
> /home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.c: In function 'iser_data_buf_aligned_len':
> /home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.c:311: warning: passing argument 2 of 'ib_sg_dma_address' makes pointer from integer without a cast
> make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.o] Error 1
> make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser] Error 2
> make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2
> make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check] Error 2
> make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5'
> make: *** [kernel] Error 2
>
> Regards,
> Vladimir
>
>
>
>