[ewg] Re: [PATCH] NFS-RDMA backport for RHEL 5.2

[Vladimir Sokolovsky]

Jon Mason wrote:
> Hey Vlad,
>
> I wanted to get you the NFS-RDMA backport patches I have queued up prior
> to rc1 being built.  I have this patch (RHEL5.2), as well as 2.6.22, and
> 2.6.25 (which I will be sending in separate e-mails).
>
> This patch contains the changes necessary to get NFS-RDMA 99% working on
> RHEL5.2.  It 100% passes of the connectathon testcases when using RDMA.
> When using TCP, 1 testcase hangs (for which I've found a code hack to
> work around, but not a fix).  The testcase is the 30MB transfer in the
> "special" tests, and is 100% reproducible.
>
> With the attached patch, you can now enable the nfs-rdma-mod by default
> and run it through your nightly build tests.
>
> There is one problem with nfs-utils, which will necessitate including it
> in the OFED distribution.  Currently, there is a check for the kernel
> version you are running when running the mount.nfs command.  This check
> is to verify if one is running a kernel greater than 2.6.22.  If not,
> then it will not allow a mount with RDMA as an option.  In earlier
> versions, there was an override option of "-i".  This has since been
> removed for an unknown reason (but if you run `mount.nfs`, you can see
> that they forgot to remove 'i' from "usage" options).  I will send them
> a patch to fix this in the latest version, but this will be a problem
> that will require it to be built locally until such a time as that is
> pulled in.
>
> In the OFED tree, there is another problem that needs to be addressed.
> There are kernel header files that are being automatically imported via
> the OFED tree.  The redefinition of these kernel data structs causes
> major problems.  Specifically, these header files are
> include/linux/pipe_fs_i.h and include/linux/splice.h.  These files
> redefine pipe_inode_info and pipe_buf_operations and many other structs
> used in OFED and the underlying kernel.  They should NOT be included,
> and can cause problems when backporting NFS-RDMA to other kernel
> versions.  I have removed their usage in my patches, but they should be
> removed from the OFED tree as they should not be needed (as we want to
> use the native header files for the kernel we are compiling against).
>
> Thanks,
> Jon
>   
Hi Jon,
Is there any reason that you removed sg_next function from 
kernel_addons/backport/2.6.18-EL5.2/include/linux/scatterlist.h?

-static inline struct scatterlist *sg_next(struct scatterlist *sg)
-{
-       if (!sg) {
-               BUG();
-               return NULL;
-       }
-       return sg + 1;
-}
-


This cause iSER compilation failure on RHEL5.2:

/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.c:253: warning: assignment makes pointer from integer without a cast
/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.c: In function 'iser_data_buf_aligned_len':
/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.c:311: warning: passing argument 2 of 'ib_sg_dma_address' makes pointer from integer without a cast
make[4]: *** [/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser/iser_memory.o] Error 1
make[3]: *** [/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband/ulp/iser] Error 2
make[2]: *** [/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check/drivers/infiniband] Error 2
make[1]: *** [_module_/home/vlad/tmp/ofa_1_4_kernel-20090302-0200_linux-2.6.18-93.el5_x86_64_check] Error 2
make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.18-93.el5'
make: *** [kernel] Error 2

Regards,
Vladimir