[ofa-general] Re: Status of NFS-RDMA ?
Krishna Kumar2
krkumar2 at in.ibm.com
Wed Jan 30 00:33:14 PST 2008
Hi James,
Since you had mentioned in an earlier email that NFS-RDMA server side will
be present in
OFED1.4, do you know if any port of the server code to OFED1.3 (when it
comes out) will
happen? Is there any effort for that, any work ongoing, any help required,
etc?
I couldn't find the release time lines for OFED1.4, is there any link on
openfabrics homepage?
Thanks,
- KK
general-bounces at lists.openfabrics.org wrote on 01/29/2008 08:23:46 PM:
>
>
> On Tue, 29 Jan 2008, Pawel Dziekonski wrote:
>
> >
> > On Mon, 28 Jan 2008 at 10:14:22AM -0500, James Lentini wrote:
> > >
> > >
> > > On Sat, 26 Jan 2008, Pawel Dziekonski wrote:
> > >
> > > > I pulled Tom's tree from new url and build a kernel.
> > >
> > > If you enabled support for INFINIBAND drivers (IB and iWARP support)
> > > and NFS client/server support, the kernel should be ready to go (run
> > > "grep RDMA /your_kernel_sources/.config" to confirm that
> > > CONFIG_SUNRPC_XPRT_RDMA is either m or y).
> > >
> > > NFS/RDMA doesn't require OFED be installed. OFED is a release of the
> > > Linux kernel sources and some userspace libraries/tools. If you are
> >
> > > > then I downloaded OFED from
> > > > http://www.mellanox.com/downloads/NFSoRDMA/OFED-1.2-NFS-RDMA.gz,
> > >
> > > I don't know what the above URL contains. The latest code is in Tom
> > > Tucker's tree (and now NFS server maintainer Bruce Fields tree). It
is
> >
> >
> > hi,
> >
> > back to subject on a proper mailing list.
> >
> > I have a >3 year experience with mellanox hardware and IBGold so I
> > basically know what OFED is all about. up to now i was only using
> > IBGold since IB drivers appeared in kernel pretty recently.
>
> You'll want to use the mainline kernel's IB drivers for NFS/RDMA.
> We've been developing the NFS/RDMA software on the OpenFabrics (aka
> OpenIB) code since it was merged into 2.6.10 in Dec 2004.
>
> > currently I have new hardware. I'm running Tom's kernel and already
> > did some MPI tests. SDP is not working, probably because sdp kernel
> > modules where not build. ;) I understand that those modules are only
> > available from ofa-kernel. please correct me if i'm wrong.
>
> Correct. SDP has never been submitted to mainline Linux.
>
> > system is Scientic Linux 4.5, which is supposed to be a fully
> > compatible RH4 clone. hardware is Supermicro mobos with Mellanox
> > MT25204 and Flextronisc switch.
> >
> > error log from ofa-kernel build:
>
> Is your goal to build a kernel with an NFS/RDMA server? If so, the
> kernel sources from Tom Tucker's git tree are the ones you want, not
> the old OFED 1.2-based packages which are out of date.
>
> Did you try setting up the NFS/RDMA server on the kernel used for your
> MPI tests above?
>
> > > > make[1]: Entering directory `/usr/src/ib/xprt-switch-2.6'
> > > > test -e include/linux/autoconf.h -a -e include/config/auto.conf ||
( \
> > > > echo; \
> > > > echo " ERROR: Kernel configuration is invalid."; \
> > > > echo " include/linux/autoconf.h or include/config/auto.conf
are
> missing."; \
> > > > echo " Run 'make oldconfig && make prepare' on kernel src
to fix it."; \
> > > > echo; \
> > > > /bin/false)
> > > >
> > > > obviously, doing 'make oldconfig && make prepare' does not help.
> > > > anyway, above mentioned files do exist:
> > > >
> > > > # ls -la /usr/src/ib/xprt-switch-2.6/{include/linux/autoconf.h,
> include/config/auto.conf}
> > > > -rw-r--r-- 1 root root 10156 Jan 25 17:42
/usr/src/ib/xprt-switch-2.
> 6/include/config/auto.conf
> > > > -rw-r--r-- 1 root root 14733 Jan 25 17:42
/usr/src/ib/xprt-switch-2.
> 6/include/linux/autoconf.h
> > > >
> > > > despite of above, compilation continues but fails with:
> > > >
> > > > gcc -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> 2/drivers/infiniband/core/.mad.o.d -nostdinc -isystem
/usr/lib/gcc/x86_64-
> redhat-linux/3.4.6/include -D__KERNEL__
-I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> 2/include -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
/drivers/infiniband/include
> -Iinclude -include include/linux/autoconf.h -include
> /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include/linux/autoconf.h -Wall
-Wundef
> -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common
-Werror-
> implicit-function-declaration -Os -m64 -mno-red-zone -mcmodel=kernel
-pipe -
> Wno-sign-compare -fno-asynchronous-unwind-tables -funit-at-a-time
-mno-sse -
> mno-mmx -mno-sse2 -mno-3dnow -maccumulate-outgoing-args -DCONFIG_AS_CFI=1
-
> DCONFIG_AS_CFI_SIGNAL_FRAME=1 -fomit-frame-pointer -Wdeclaration-after-
> statement -DMODULE -D"KBUILD_STR(s)=#s" -
> D"KBUILD_BASENAME=KBUILD_STR(mad)" -D"KBUILD_MODNAME=KBUILD_STR(ib_mad)"
-c -
> o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/core/.!
> tmp
> > _mad.o /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
/drivers/infiniband/core/mad.c
> > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
/drivers/infiniband/core/mad.c: In
> function `ib_mad_init_module':
> > > > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
/drivers/infiniband/core/mad.c:
> 2966: error: too many arguments to function `kmem_cache_create'
> > > > make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> 2/drivers/infiniband/core/mad.o] Error 1
> > > > make[3]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.
> 2/drivers/infiniband/core] Error 2
> > > > make[2]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2
/drivers/infiniband] Error 2
> > > > make[1]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2] Error
2
> > > > make[1]: Leaving directory `/usr/src/ib/xprt-switch-2.6'
> > > > make: *** [kernel] Error 2
> > > > error: Bad exit status from /var/tmp/rpm-tmp.3877 (%install)
> >
> > > > full log:
> > > > https://cefeid.wcss.wroc.pl/d/tmp/OFED.build.32122.log
> >
> > thanks in advance for any help, P
> >
> >
> > --
> > Pawel Dziekonski <pawel.dziekonski at wcss.pl>
> > Wroclaw Centre for Networking & Supercomputing, HPC Department
> > Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw,
POLAND
> > phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl
> >
> >
-------------------------------------------------------------------------
> > This SF.net email is sponsored by: Microsoft
> > Defy all challenges. Microsoft(R) Visual Studio 2008.
> > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > _______________________________________________
> > nfs-rdma-devel mailing list
> > nfs-rdma-devel at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/nfs-rdma-devel
> >
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-general
More information about the general
mailing list