[ofa-general] [NFS/RDMA] Can't mount NFS/RDMA partition

[Jeff Becker]

Hi Celine

Celine Bourde wrote:
> Hi,
>
> I can't mount an NFS/RDMA partition. 
>
> I've a linux 2.6.27 kernel with ofa_kernel-1.4.1 (provided by OFED-1.4.1-rc3).
> [root at host]# modinfo xprtrdma
> filename:       /lib/modules/2.6.27_ofa_compil/updates/kernel/net/sunrpc/xprtrdma/xprtrdma.ko
> [root at thost]# modinfo svcrdma
> filename:       /lib/modules/2.6.27_ofa_compil/updates/kernel/net/sunrpc/xprtrdma/svcrdma.ko
> [..]
>
> I've applied
> http://www.openfabrics.org//downloads/OFED/ofed-1.4/OFED-1.4-docs/nfs-rdma.release-notes.txt
> instructions.
>
> Every steps (loading modules, /etc/exports implementation, starting nfs daemon,
> etc..) seems to be ok, but when I do the last command (nfs-utils-1.1.4):
> mount -o rdma,port=2050 192.168.0.215:/export /tmp/nfs_client/
> The mount processus blocks. My output is the following  :
> "
> rpcrdma: connection to 192.168.0.215:2050 on mlx4_0, memreg 5 slots 32 ird 16
> rpcrdma: connection to 192.168.0.215:2050 closed (-103)
> rpcrdma: connection to 192.168.0.215:2050 on mlx4_0, memreg 5 slots 32 ird 16
> rpcrdma: connection to 192.168.0.215:2050 closed (-103)
> rpcrdma: connection to 192.168.0.215:2050 on mlx4_0, memreg 5 slots 32 ird 16
> rpcrdma: connection to 192.168.0.215:2050 closed (-103)
> rpcrdma: connection to 192.168.0.215:2050 on mlx4_0, memreg 5 slots 32 ird 16"
>   

On the server did you remember to:

echo rdma 2050 > /proc/fs/nfsd/portlist


? Also, make sure that none of the stock 2.6.27 NFS modules are loaded -
they should all
be from OFED 1.4.1

-jeff

> [root at host]#strace mount -o rdma,port=2050 192.168.0.215:/vol0 /mnt
> [...]
> read(3, "# This file controls the state o"..., 4096) = 447
> read(3, "", 4096)                       = 0
> close(3)                                = 0
> munmap(0x7f7268080000, 4096)            = 0
> open("/proc/mounts", O_RDONLY)          = 3
> fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7268080000
> read(3, "rootfs / rootfs rw 0 0\n/dev/root"..., 1024) = 693
> read(3, "", 1024)                       = 0
> close(3)                                = 0
> munmap(0x7f7268080000, 4096)            = 0
> open("/usr/lib/locale/locale-archive", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=56405456, ...}) = 0
> mmap(NULL, 56405456, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f7264a99000
> close(3)                                = 0
> umask(022)                              = 022
> open("/dev/null", O_RDWR)               = 3
> close(3)                                = 0
> getuid()                                = 0
> geteuid()                               = 0
> getgid()                                = 0
> getegid()                               = 0
> prctl(0x3, 0, 0, 0, 0)                  = 1
> open("/etc/blkid/blkid.tab", O_RDONLY)  = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=643, ...}) = 0
> fcntl(3, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)
> fstat(3, {st_mode=S_IFREG|0644, st_size=643, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7268080000
> lseek(3, 0, SEEK_CUR)                   = 0
> read(3, "<device DEVNO=\"0x0900\" TIME=\"124"..., 4096) = 643
> read(3, "", 4096)                       = 0
> close(3)                                = 0
> munmap(0x7f7268080000, 4096)            = 0
> getuid()                                = 0
> geteuid()                               = 0
> lstat("/etc/mtab", {st_mode=S_IFREG|0644, st_size=296, ...}) = 0
> stat("192.168.0.215:/vol0", 0x7fff700810c0) = -1 ENOENT (No such file or directory)
> stat("/sbin/mount.nfs", {st_mode=S_IFREG|S_ISUID|0511, st_size=287532, ...}) = 0
> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f72680647f0) = 4991
> wait4(-1, 
>
> and it blocks.
>
> Any Idea ?
>
> Thanks for your help.
>
> Céline Bourde.
>
>
>
>