[ewg] NFS/RDMA backport for OFED 4.8-1 part 3
Stefan Oesterreich
soesterreich at iol.unh.edu
Tue Sep 5 13:47:28 PDT 2017
Hi All,
I have loaded the daily build on our PPC64LE box here at the UNH-IOL. I did
reboot the machine after OFED install. It looks like it is loading fine, it
loads without errors:
[soesterreich at hyrrokkin:~]$sudo modprobe rpcrdma
[soesterreich at hyrrokkin:~]$
[soesterreich at hyrrokkin:~]$lsmod | grep rpcrdma
rpcrdma 106978 0
rdma_cm 61373 2 rpcrdma,rdma_ucm
ib_core 253189 15
iw_cxgb3,iw_cxgb4,rdma_cm,ib_cm,iw_cm,rpcrdma,mlx4_ib,mlx5_ib,ib_ucm,iw_nes,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib
compat 13726 20
iw_cxgb3,iw_cxgb4,rdma_cm,cxgb3,cxgb4,ib_cm,iw_cm,rpcrdma,mlx4_en,mlx4_ib,mlx5_ib,ib_ucm,iw_nes,ib_core,ib_umad,ib_uverbs,mlx4_core,mlx5_core,rdma_ucm,ib_ipoib
sunrpc 468639 16
nfs,nfsd,rpcsec_gss_krb5,auth_rpcgss,lockd,nfsv4,rpcrdma,nfs_acl
Looks like he just forgot to reboot. Thank you for your assistance.
--Stefan
On Tue, Sep 5, 2017 at 4:43 PM, David Z. Dai <zdai at linux.vnet.ibm.com>
wrote:
> Hi, Vlad:
> No, I didn't reboot the server after a successful OFED installation,
> and I didn't restart the openibd either. I don't remember I did this
> with OFED-3.18 release.
>
> I follow your suggestion, did a reboot after the installation. Now it
> works fine!
> [root at rhel73_host1_guest1_local_htx ~]# modprobe rpcrdma
> [root at rhel73_host1_guest1_local_htx ~]#
> [root at rhel73_host1_guest1_local_htx ~]# lsmod |grep rpcrdma
> rpcrdma 106978 0
>
> [root at rhel73_host1_guest1_local_htx ~]# modprobe -r rpcrdma
> [root at rhel73_host1_guest1_local_htx ~]#
>
> I haven't tested the functionality yet. At least module is loading fine
> now.
>
> Thanks for the help! - David Dai
>
> On Tue, 2017-09-05 at 23:18 +0300, Vladimir Sokolovsky wrote:
> > Hi David,
> > Did you reboot the server after OFED installation or reloaded drivers
> > with "/etc/init.d/openibd restart" command?
> >
> > Regards,
> > Vladimir
> >
> > On 09/05/2017 10:51 PM, David Z. Dai wrote:
> > > Hi, Jeff:
> > > It has the same error when loading "rpmcrdma" directly.
> > > [root at rhel73_host1_guest1_local_htx modules]# modprobe rpcrdma
> > > modprobe: ERROR: could not insert 'rpcrdma': Exec format error
> > >
> > > I also change the "rpcrdma.ko" mode from 644 to 744. It still has the
> > > same issue.
> > >
> > > Thanks! - David Dai
> > >
> > > On Tue, 2017-09-05 at 12:33 -0700, Jeff Becker wrote:
> > >> Hi,
> > >>
> > >> I responded to the bugzilla 2633. In short, please try "modprobe
> > >> rpcrdma" directly, and just once as it now contains both server and
> > >> client. Thanks.
> > >>
> > >> -jeff
> > >>
> > >> On 09/05/2017 12:25 PM, David Z. Dai wrote:
> > >>> Hi, Vlad, Jeff:
> > >>> Today I got the latest daily build from:
> > >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__
> downloads.openfabrics.org_OFED_ofed-2D4.8-2D1-2Ddaily_
> OFED-2D4.8-2D1-2D20170905-2D0757.tgz&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=
> jE7tivF653Vygp552zF5nqagPKJAuy1x71Qnyf3szIw&m=g18gQ6E3E67ica6Ua-
> VcSfLqUhqoidoQ6p9JBZX16ZM&s=7P1W9rvFy-ZeD1z4hfqzovK40umK372kBX0EKXyeYfQ&e=
> > >>> This latest daily build includes the NFSoRDMA functionality now.
> > >>>
> > >>> I did a full install on RHEL 7.3 LE OS:
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]#
> uname -a
> > >>> Linux rhel73_host1_guest1_local_htx 3.10.0-514.el7.ppc64le #1 SMP
> Wed Oct 19
> > >>> 11:27:06 EDT 2016 ppc64le ppc64le ppc64le GNU/Linux
> > >>>
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ./
> install.pl --all
> > >>>
> > >>> The installation finished successfully.
> > >>>
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]#
> ofed_info -s
> > >>> OFED-4.8-1-20170905-0757:
> > >>>
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]#
> modinfo xprtrdma
> > >>> filename: /lib/modules/3.10.0-514.el7.
> ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
> > >>> alias: xprtrdma
> > >>> alias: svcrdma
> > >>> license: Dual BSD/GPL
> > >>> description: RPC/RDMA Transport
> > >>> author: Open Grid Computing and Network Appliance, Inc.
> > >>> rhelversion: 7.3
> > >>> srcversion: 9629610592FC8363A00B592
> > >>> depends: sunrpc,compat,ib_core,rdma_cm
> > >>> vermagic: 3.10.0-514.el7.ppc64le SMP mod_unload modversions
> > >>>
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]#
> modinfo svcrdma
> > >>> filename: /lib/modules/3.10.0-514.el7.
> ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
> > >>> alias: xprtrdma
> > >>> alias: svcrdma
> > >>> license: Dual BSD/GPL
> > >>> description: RPC/RDMA Transport
> > >>> author: Open Grid Computing and Network Appliance, Inc.
> > >>> rhelversion: 7.3
> > >>> srcversion: 9629610592FC8363A00B592n
> > >>> depends: sunrpc,compat,ib_core,rdma_cm
> > >>> vermagic: 3.10.0-514.el7.ppc64le SMP mod_unload modversions
> > >>>
> > >>> They are not in "nfsrdma" rpm any more. They are now in
> "compat-rdma" rpm:
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# rpm
> -qf
> > >>> /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/
> xprtrdma/rpcrdma.ko
> > >>> compat-rdma-4.8-0.20170905.0757.g63258b0.3.10.0_514.el7.
> ppc64le.ppc64le
> > >>>
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# rpm
> -qf
> > >>> /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/
> xprtrdma/rpcrdma.ko
> > >>> compat-rdma-4.8-0.20170905.0757.g63258b0.3.10.0_514.el7.
> ppc64le.ppc64le
> > >>>
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ls
> -al /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/
> xprtrdma/rpcrdma.ko
> > >>> -rw-r--r--. 1 root root 6031120 Sep 5 14:24
> /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ls
> -al /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/
> xprtrdma/rpcrdma.ko
> > >>> -rw-r--r--. 1 root root 6031120 Sep 5 14:24
> /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
> > >>>
> > >>> However I see the error during module load time:
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]#
> modprobe xprtrdma
> > >>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error
> > >>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]#
> modprobe svcrdma
> > >>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error
> > >>>
> > >>>
> > >>> NOTE:
> > >>> 1) Both xprtrdma and svcrdma are using same "rpcrdma.ko" (see
> modinfo output above)
> > >>> 2) The mode of "rpcrdma.ko" is 644. Should it be at least 744?
> > >>>
> > >>> Could you please check on this?
> > >>>
> > >>> Thanks! - David
> > >>> On Sat, 2017-09-02 at 00:59 +0300, Vladimir Sokolovsky wrote:
> > >>>> Applied + build: OFED-4.8-1-20170901-1439
> > >>>>
> > >>>> Regards,
> > >>>> Vladimir
> > >>>>
> > >>>> On 09/02/2017 12:44 AM, Jeff Becker wrote:
> > >>>>> Hi Vlad,
> > >>>>>
> > >>>>> Please apply the attached patches to compat-rdma.git. Together with
> > >>>>> the previous two patches, these have been tested with RHEL7.3 and
> > >>>>> SLES12SP2. Thanks.
> > >>>>>
> > >>>>> -jeff
> > >>>>>
> > >>>> _______________________________________________
> > >>>> ewg mailing list
> > >>>> ewg at lists.openfabrics.org
> > >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.
> openfabrics.org_mailman_listinfo_ewg&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=
> jE7tivF653Vygp552zF5nqagPKJAuy1x71Qnyf3szIw&m=antYpg37AAv4LKbRqOFw77idqtt-
> Kxw8KtFSKz_3UJE&s=RNm5Hv7MRR_y-c_JC9pR1ZHCbLAkAtQh5HJCBEOEfj0&e=
> > >
> >
>
>
>
--
*-----------------------------------------------Cheers,Stefan Oesterreich*
*High Performance Computing*
*UNH InterOperability
Laboratory------------------------------------------------*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20170905/bbcb2166/attachment.html>
More information about the ewg
mailing list