[nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ?
Pawel Dziekonski
pawel.dziekonski at pwr.wroc.pl
Mon Feb 4 07:28:58 PST 2008
On Mon, 04 Feb 2008 at 09:59:50AM -0500, James Lentini wrote:
>
>
> On Fri, 1 Feb 2008, Pawel Dziekonski wrote:
>
> > On Fri, 01 Feb 2008 at 10:56:55AM -0500, James Lentini wrote:
> > > > # mount 10.2.2.1:/scratch /mnt -i -o rdma,port=2050 Unsupported nfs
> > > > mount option: rdma
> > > >
> > > > looks like I definitelly need a rdma-enabled mount, which comes in
> > > > http://www.mellanox.com/downloads/NFSoRDMA/OFED-1.2-NFS-RDMA.gz, so:
> > >
> > > If you are using the NFS/RDMA client in 2.6.24, you need the version
> > > of the mount.nfs command in nfs-utils-1.1.1 or greater.
> > >
> > > The "Unsupported nfs mount option: rdma" error message makes me
> > > suspect you are not using the correct version of mount.nfs. What is
> > > the output of "mount.nfs -V" and "mount -V"? It may be that the
> > > version of mount you are using does not automatically invoke mount.nfs
> > > for nfs mounts. I'd suggest specifying mount.nfs in the command above:
> >
> > # mount.nfs -V
> > mount.nfs (linux nfs-utils 1.1.1)
> >
> > > mount.nfs 10.2.2.1:/scratch /mnt -i -o rdma,port=2050
> >
> > # mount.nfs 10.2.2.1:/scratch /mnt -i -o rdma,port=2050 -v
> > mount.nfs: timeout set for Fri Feb 1 23:48:08 2008
> > mount.nfs: text-based options: 'rdma,port=2050,addr=10.2.2.1'
> > mount.nfs: internal error
> >
> > :(
>
> I'm going to guess that the "internal error" message means that your
> kernel does not support the NFS string mount API. Are you sure your
> kernel you are using is an exact copy of Tom Tucker's git tree? It
> can NOT have the OFED 1.2 package installed.
kernel was pulled from Tom's tree, Jan 25th.
# cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
url = git://git.linux-nfs.org/projects/tomtucker/xprt-switch-2.6.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "merged-fixes"]
remote = origin
merge = refs/heads/merged-fixes
config: https://cefeid.wcss.wroc.pl/d/tmp/c-2.6.24-rc6
What do you mean 'It can NOT have the OFED 1.2 package installed.'?
I can not install OFED 1.2 on this machine at all?!
> If the kernel is correct, can you send the output of:
>
> strace -e trace=all 10.2.2.1:/scratch /mnt -i -o rdma,port=2050
execve("/sbin/mount.nfs", ["mount.nfs", "10.2.2.1:/scratch", "/mnt", "-i", "-o", "rdma,port=2050"], [/* 28 vars */]) = 0
uname({sys="Linux", node="ib2", ...}) = 0
brk(0) = 0x613000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b68b8a78000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/opt/intel/mkl/10.0.1.014/lib/em64t/tls/x86_64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/mkl/10.0.1.014/lib/em64t/tls/x86_64", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/mkl/10.0.1.014/lib/em64t/tls/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/mkl/10.0.1.014/lib/em64t/tls", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/mkl/10.0.1.014/lib/em64t/x86_64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/mkl/10.0.1.014/lib/em64t/x86_64", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/mkl/10.0.1.014/lib/em64t/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/mkl/10.0.1.014/lib/em64t", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/opt/intel/fce/10.1.011/lib/tls/x86_64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/fce/10.1.011/lib/tls/x86_64", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/fce/10.1.011/lib/tls/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/fce/10.1.011/lib/tls", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/fce/10.1.011/lib/x86_64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/fce/10.1.011/lib/x86_64", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/fce/10.1.011/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/fce/10.1.011/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/opt/intel/cce/10.1.011/lib/tls/x86_64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/cce/10.1.011/lib/tls/x86_64", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/cce/10.1.011/lib/tls/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/cce/10.1.011/lib/tls", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/cce/10.1.011/lib/x86_64/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/cce/10.1.011/lib/x86_64", 0x7ffff202db10) = -1 ENOENT (No such file or directory)
open("/opt/intel/cce/10.1.011/lib/libc.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/opt/intel/cce/10.1.011/lib", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=38220, ...}) = 0
mmap(NULL, 38220, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b68b8a79000
close(3) = 0
open("/lib64/tls/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\304"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1622600, ...}) = 0
mmap(0x33f2600000, 2314184, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x33f2600000
mprotect(0x33f272c000, 1085384, PROT_NONE) = 0
mmap(0x33f282c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x12c000) = 0x33f282c000
mmap(0x33f2831000, 16328, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x33f2831000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b68b8a83000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b68b8a84000
mprotect(0x33f282c000, 12288, PROT_READ) = 0
mprotect(0x33f2514000, 4096, PROT_READ) = 0
arch_prctl(ARCH_SET_FS, 0x2b68b8a83b00) = 0
munmap(0x2b68b8a79000, 38220) = 0
getuid() = 0
uname({sys="Linux", node="ib2", ...}) = 0
uname({sys="Linux", node="ib2", ...}) = 0
brk(0) = 0x613000
brk(0x634000) = 0x634000
lstat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
access("/mnt", X_OK) = 0
mount("10.2.2.1:/scratch", "/mnt", "nfs", 0, "rdma,port=2050,addr=10.2.2.1") = -1 EIO (Input/output error)
write(2, "mount.nfs: internal error\n", 26mount.nfs: internal error
) = 26
exit_group(32) = ?
Process 32607 detached
--
Pawel Dziekonski <pawel.dziekonski at wcss.pl>
Wroclaw Centre for Networking & Supercomputing, HPC Department
Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND
phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl
More information about the general
mailing list