[Fwd: Re: [ofa-general] [NFS/RDMA] Can't mount NFS/RDMA partition]]

[Steve Wise]

Celine Bourde wrote:
> There was a mistake in my last email.
> My last nfs mount test with no rdma was not correct, I've retried with
>
> [root at my_host ] #mount -o rw 192.168.0.215:/vol0 /mnt/
>
> My nfs partition is mounted and everything is correct, which confirms 
> that the problem comes from rdma connection manager.
>
> Céline.
>

Can you run rping or one of the perf programs over the IB link?  This 
will confirm that your IB setup works.

Also, on the nfs server, what is the output of 'cat /proc/fs/nfsd/portlist'?


>
> Celine Bourde wrote:
>
>> Hi,
>>  
>> I've updated nfs-utils package:
>>   [root at my_host ~]# mount.nfs -V
>>   mount.nfs (linux nfs-utils 1.1.6)
>>    
>>> [root at my_host ~]# strace  mount.nfs
>>> 192.168.0.215:/vol0 /mnt/ -o rdma,port=2050
>>>     Does it work without rdma?
>>>     
>>
>> The problem is exactly the same without rdma:
>>  
>> [root at my_host ~]# strace  mount.nfs 192.168.0.215:/vol0 /mnt/ -o
>> rw,port=2050
>>   [..]
>>   socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 3
>>   bind(3, {sa_family=AF_INET, sin_port=htons(0),
>> sin_addr=inet_addr("0.0.0.0")}, 16) = 0
>>   fcntl(3, F_GETFL)                       = 0x2 (flags O_RDWR)
>>   fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
>>   connect(3, {sa_family=AF_INET, sin_port=htons(997),
>> sin_addr=inet_addr("127.0.0.1")}, 16) = 0
>>   fcntl(3, F_SETFL, O_RDWR)               = 0
>>   connect(3, {sa_family=AF_UNSPEC,
>> sa_data="\0o\177\0\0\1\0\0\0\0\0\0\0\0"}, 16) = 0
>>   sendto(3,
>> "\0308\310\272\0\0\0\0\0\0\0\2\0\1\206\270\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0"..., 
>>
>> 40, 0, {sa_family=AF_INET, sin_port=htons(997),
>> sin_addr=inet_addr("127.0.0.1")}, 16) = 40
>>   poll([{fd=3, events=POLLIN}], 1, 3000)  = 1 ([{fd=3, revents=POLLIN}])
>>   recvfrom(3, "\0308\310\272\0\0\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0",
>> 400, MSG_DONTWAIT, {sa_family=AF_INET, sin_port=htons(997),
>> sin_addr=inet_addr("127.0.0.1")}, [16]) = 24
>>   close(3)                                = 0
>>   mount("192.168.0.215:/vol0", "/mnt", "nfs", 0,
>> "port=2050,addr=192.168.0.215"
>>  
>> .. and it blocks !
>>  
>> [root at my_host ~]# dmesg
>>   rpcrdma: connection to 192.168.0.215:2050 on mlx4_0, memreg 5 slots 32
>> ird 16
>>   rpcrdma: connection to 192.168.0.215:2050 closed (-103)
>>   rpcrdma: connection to 192.168.0.215:2050 on mlx4_0, memreg 5 slots 32
>> ird 16
>>   rpcrdma: connection to 192.168.0.215:2050 closed (-103)
>>    
>>> Celine,  can you establish regular rdma
>>> connections over your IB link?  Like rping?
>>>     
>>
>> I can't etablish rdma connection, following errors occur with rping :
>>  
>> [root at my_host ~]# rping -c 192.168.0.214
>>   cq completion failed status 5
>>   wait for CONNECTED state 10
>>   connect error -1
>>   cma event RDMA_CM_EVENT_REJECTED, error 8
>>  
>> My kernel is a 2.6.27 kernel.org build on a Red Hat Distribution : Red
>> Hat Enterprise Linux Server release 5.3 Beta (Tikanga).
>>     Céline.
>>  
>> _______________________________________________
>>   general mailing list
>>   general at lists.openfabrics.org
>>   http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>  
>> To unsubscribe, please visit
>> http://openib.org/mailman/listinfo/openib-general
>>