[nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ?

[Pawel Dziekonski]

On Mon, 04 Feb 2008 at 05:39:25PM -0500, James Lentini wrote:

> I don't use OFED, but I'll share with you what I know. OFED 1.3 is 
> based on 2.6.24, which is the same kernel Tom Tucker's git tree is 
> based on. I would assume the two are compatible, but I have not tried 
> it.
> 
> Do you have NFS/RDMA working at this point?

no - internal error again. :(

I have uninstalled OFED completely and I'm running pure Tom Tucker's
kernel.

yum install opensm
/etc/init.d/opensmd start
cat /sys/class/infiniband/mthca0/ports/1/state
4: ACTIVE

ifconfig ib0
ib0       Link encap:UNSPEC  HWaddr 80-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00
          inet addr:10.2.2.1  Bcast:10.2.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:128
          RX bytes:784 (784.0 b)  TX bytes:552 (552.0 b)

rpm -e nfs-utils
cd nfs-utils-1.1.1/
./configure --prefix=/usr/local --disable-gss --disable-nfs4
make
make install
cd ..

cat /etc/exports
/scratch 10.255.255.222(no_subtree_check,rw,async) 10.2.2.2(no_subtree_check,rw,async,insecure,no_root_squash)

cat NFS_START_SERVER
#!/bin/sh
export PATH="/usr/local/sbin:/usr/local/bin:$PATH"
/etc/rc.d/init.d/portmap restart
modprobe svcrdma
modprobe nfs
umount /proc/fs/nfsd >&/dev/null
mount -t nfsd nfsd /proc/fs/nfsd >&/dev/null
mount |grep /proc/fs/nfsd
exportfs -av
rpc.mountd
rpc.statd --no-notify
rpc.nfsd
sm-notify
echo rdma 2050 > /proc/fs/nfsd/portlist

./NFS_START_SERVER

same done on client.

cat NFS_START_CLIENT
#!/bin/sh
export PATH="/usr/local/sbin:/usr/local/bin:$PATH"
/etc/rc.d/init.d/portmap restart
modprobe svcrdma
modprobe nfs
sm-notify

./NFS_START_CLIENT

mount.nfs 10.2.2.1:/scratch /mnt -i -o rdma,port=2050 -v     
mount.nfs: timeout set for Tue Feb  5 14:50:47 2008
mount.nfs: text-based options: 'rdma,port=2050,addr=10.2.2.1'
mount.nfs: internal error

mount("10.2.2.1:/scratch", "/mnt", "nfs", 0, "rdma,port=2050,addr=10.2.2.1") = -1 EIO (Input/output error)
write(2, "mount.nfs: internal error\n", 26mount.nfs: internal error
) = 26
exit_group(32)                          = ?
Process 13170 detached

server logs:
Feb  5 14:45:46 ib1 mountd[19614]: authenticated mount request from 10.2.2.2:988 for /scratch (/scratch)

:(

-- 
Pawel Dziekonski <pawel.dziekonski at wcss.pl>
Wroclaw Centre for Networking & Supercomputing, HPC Department
Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND
phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl