[nfs-rdma-devel] [ofa-general] Status of NFS-RDMA ?

[Pawel Dziekonski]

On Tue, 05 Feb 2008 at 10:04:46AM -0500, James Lentini wrote:

> > ifconfig ib0
> > ib0       Link encap:UNSPEC  HWaddr 80-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00
> >           inet addr:10.2.2.1  Bcast:10.2.2.255  Mask:255.255.255.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
> >           RX packets:8 errors:0 dropped:0 overruns:0 frame:0
> >           TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:128
> >           RX bytes:784 (784.0 b)  TX bytes:552 (552.0 b)
> 
> Is this ib0 interface on the server?

yes.

> Can you ping the server's IPoIB interface from your client?
> Based on the output below, I think to these questions is yes.

yes.

# ping -c 3 10.2.2.2
PING 10.2.2.2 (10.2.2.2) 56(84) bytes of data.
64 bytes from 10.2.2.2: icmp_seq=0 ttl=64 time=1.91 ms
64 bytes from 10.2.2.2: icmp_seq=1 ttl=64 time=0.033 ms
64 bytes from 10.2.2.2: icmp_seq=2 ttl=64 time=0.025 ms

--- 10.2.2.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.025/0.657/1.915/0.889 ms, pipe 2

> > rpm -e nfs-utils
> > cd nfs-utils-1.1.1/
> > ./configure --prefix=/usr/local --disable-gss --disable-nfs4
> > make
> > make install
> > cd ..

> > same done on client.
> 
> What is the same on the client?

nfs-utils installed in the very same way.

> > modprobe svcrdma
> 
> You don't need to load the svcrdma module on the client. The svcrdma 
> module is only needed on the server.
> 
> The client needs to have the xprtrdma code loaded. You appear to have 
> built nfs as a module, so the xprtrdma code would also have been built 
> as a module. Instead of loading scvrdma, you should run this here:
> 
>  modprobe xprtrdma

ok!

> Did you see this message on the client? 
>  RPC: transport (256) not supported

no.

> If loading the xprtrdma module doesn't solve your problem, send the 
> output from these commands:
> 
>  dmesc -c > /dev/null
>  echo 1024 > /proc/sys/sunrpc/nfs_debug
>  mount.nfs 10.2.2.1:/scratch /mnt -i -o rdma,port=2050 -v
>  dmesg > output

# modprobe xprtrdma
# mount.nfs 10.2.2.1:/scratch /mnt -i -o rdma,port=2050 -v
mount.nfs 10.2.2.1:/scratch /mnt -i -o rdma,port=2050 -v
mount.nfs: timeout set for Tue Feb  5 16:31:19 2008
mount.nfs: text-based options: 'rdma,port=2050,addr=10.2.2.1'
mount.nfs: internal error
# dmesg
NFS: nfs mount opts='rdma,port=2050,addr=10.2.2.1'
NFS:   parsing nfs mount option 'rdma'
NFS:   parsing nfs mount option 'port=2050'
NFS:   parsing nfs mount option 'addr=10.2.2.1'
NFS: sending MNT request for 10.2.2.1:/scratch
NFS: MNT request succeeded

-- 
Pawel Dziekonski <pawel.dziekonski at wcss.pl>
Wroclaw Centre for Networking & Supercomputing, HPC Department
Politechnika Wr., pl. Grunwaldzki 9, bud. D2/101, 50-377 Wroclaw, POLAND
phone: +48 71 3202043, fax: +48 71 3225797, http://www.wcss.wroc.pl