[ofa-general] Re: Status of NFS-RDMA ?

James Lentini jlentini at netapp.com
Wed Jan 30 06:27:21 PST 2008 


On Wed, 30 Jan 2008, Krishna Kumar2 wrote:

> Hi James,
> 
> Since you had mentioned in an earlier email that NFS-RDMA server 
> side will be present in OFED1.4, 

Actually, that was Tziporet.

> do you know if any port of the server code to OFED1.3 (when it comes 
> out) will happen? Is there any effort for that, any work ongoing, 
> any help required, etc?

Jeff Becker had looked into this. We would definitely appreciate the 
help.

The NFS framework has changed significantly in several areas in recent 
kernel releases. This has made backporting the NFS/RDMA code to older 
kernels challenging.

If you are interested in working on OFED1.3 support, let us know.

> I couldn't find the release time lines for OFED1.4, is there any 
> link on openfabrics homepage?

I'm not involved with the OFED1.4 planning. Tziporet, is there 
information on this?

> Thanks,
> 
> - KK
> 
> general-bounces at lists.openfabrics.org wrote on 01/29/2008 08:23:46 PM:
> 
> >
> >
> > On Tue, 29 Jan 2008, Pawel Dziekonski wrote:
> >
> > >
> > > On Mon, 28 Jan 2008 at 10:14:22AM -0500, James Lentini wrote:
> > > >
> > > >
> > > > On Sat, 26 Jan 2008, Pawel Dziekonski wrote:
> > > >
> > > > > I pulled Tom's tree from new url and build a kernel.
> > > >
> > > > If you enabled support for INFINIBAND drivers (IB and iWARP support)
> > > > and NFS client/server support, the kernel should be ready to go (run
> > > > "grep RDMA /your_kernel_sources/.config" to confirm that
> > > > CONFIG_SUNRPC_XPRT_RDMA is either m or y).
> > > >
> > > > NFS/RDMA doesn't require OFED be installed. OFED is a release of the
> > > > Linux kernel sources and some userspace libraries/tools. If you are
> > >
> > > > > then I downloaded OFED from
> > > > > http://www.mellanox.com/downloads/NFSoRDMA/OFED-1.2-NFS-RDMA.gz,
> > > >
> > > > I don't know what the above URL contains. The latest code is in Tom
> > > > Tucker's tree (and now NFS server maintainer Bruce Fields tree). It
> is
> > >
> > >
> > > hi,
> > >
> > > back to subject on a proper mailing list.
> > >
> > > I have a >3 year experience with mellanox hardware and IBGold so I
> > > basically know what OFED is all about. up to now i was only using
> > > IBGold since IB drivers appeared in kernel pretty recently.
> >
> > You'll want to use the mainline kernel's IB drivers for NFS/RDMA.
> > We've been developing the NFS/RDMA software on the OpenFabrics (aka
> > OpenIB) code since it was merged into 2.6.10 in Dec 2004.
> >
> > > currently I have new hardware. I'm running Tom's kernel and already
> > > did some MPI tests. SDP is not working, probably because sdp kernel
> > > modules where not build. ;) I understand that those modules are only
> > > available from ofa-kernel. please correct me if i'm wrong.
> >
> > Correct. SDP has never been submitted to mainline Linux.
> >
> > > system is Scientic Linux 4.5, which is supposed to be a fully
> > > compatible RH4 clone. hardware is Supermicro mobos with Mellanox
> > > MT25204 and Flextronisc switch.
> > >
> > > error log from ofa-kernel build:
> >
> > Is your goal to build a kernel with an NFS/RDMA server? If so, the
> > kernel sources from Tom Tucker's git tree are the ones you want, not
> > the old OFED 1.2-based packages which are out of date.
> >
> > Did you try setting up the NFS/RDMA server on the kernel used for your
> > MPI tests above?
> >
> > > > > make[1]: Entering directory `/usr/src/ib/xprt-switch-2.6'
> > > > > test -e include/linux/autoconf.h -a -e include/config/auto.conf ||
> (            \
> > > > > echo;                                                           \
> > > > > echo "  ERROR: Kernel configuration is invalid.";               \
> > > > > echo "         include/linux/autoconf.h or include/config/auto.conf
> are
> > missing.";      \
> > > > > echo "         Run 'make oldconfig && make prepare' on kernel src
> to fix it.";  \
> > > > > echo;                                                           \
> > > > > /bin/false)
> > > > >
> > > > > obviously, doing 'make oldconfig && make prepare' does not help.
> > > > > anyway, above mentioned files do exist:
> > > > >
> > > > > # ls -la /usr/src/ib/xprt-switch-2.6/{include/linux/autoconf.h,
> > include/config/auto.conf}
> > > > > -rw-r--r--  1 root root 10156 Jan 25 17:42
> /usr/src/ib/xprt-switch-2.
> > 6/include/config/auto.conf
> > > > > -rw-r--r--  1 root root 14733 Jan 25 17:42
> /usr/src/ib/xprt-switch-2.
> > 6/include/linux/autoconf.h
> > > > >
> > > > > despite of above, compilation continues but fails with:
> > > > >
> > > > >   gcc -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> > 2/drivers/infiniband/core/.mad.o.d  -nostdinc -isystem
> /usr/lib/gcc/x86_64-
> > redhat-linux/3.4.6/include -D__KERNEL__
> -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> > 2/include  -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
> /drivers/infiniband/include
> > -Iinclude    -include include/linux/autoconf.h  -include
> > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include/linux/autoconf.h   -Wall
> -Wundef
> > -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common
> -Werror-
> > implicit-function-declaration -Os   -m64 -mno-red-zone -mcmodel=kernel
> -pipe -
> > Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time
> -mno-sse -
> > mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1
> -
> > DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer  -Wdeclaration-after-
> > statement     -DMODULE -D"KBUILD_STR(s)=#s" -
> > D"KBUILD_BASENAME=KBUILD_STR(mad)"  -D"KBUILD_MODNAME=KBUILD_STR(ib_mad)"
> -c -
> > o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/.!
> >  tmp
> > >  _mad.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
> /drivers/infiniband/core/mad.c
> > > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
> /drivers/infiniband/core/mad.c: In
> > function `ib_mad_init_module':
> > > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
> /drivers/infiniband/core/mad.c:
> > 2966: error: too many arguments to function `kmem_cache_create'
> > > > > make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> > 2/drivers/infiniband/core/mad.o] Error 1
> > > > > make[3]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> > 2/drivers/infiniband/core] Error 2
> > > > > make[2]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
> /drivers/infiniband] Error 2
> > > > > make[1]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2] Error
> 2
> > > > > make[1]: Leaving directory `/usr/src/ib/xprt-switch-2.6'
> > > > > make: *** [kernel] Error 2
> > > > > error: Bad exit status from /var/tmp/rpm-tmp.3877 (%install)
> > >
> > > > > full log:
> > > > > https://cefeid.wcss.wroc.pl/d/tmp/OFED.build.32122.log
> > >
> > > thanks in advance for any help, P
> > >
> > >
> > > --
> > > Pawel Dziekonski <pawel.dziekonski at wcss.pl>
> > > Wroclaw Centre for Networking & Supercomputing, HPC Department
> > > Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw,
> POLAND
> > > phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl
> > >
> > >
> -------------------------------------------------------------------------
> > > This SF.net email is sponsored by: Microsoft
> > > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > > _______________________________________________
> > > nfs-rdma-devel mailing list
> > > nfs-rdma-devel at lists.sourceforge.net
> > > https://lists.sourceforge.net/lists/listinfo/nfs-rdma-devel
> > >
> > _______________________________________________
> > general mailing list
> > general at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >
> > To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>

More information about the general mailing list