<div dir="ltr"><div class="gmail_default" style="font-size:large"><div class="gmail_default" style="font-size:large">Hi All,</div><div class="gmail_default" style="font-size:large"><br></div><div class="gmail_default" style="font-size:large">I have loaded the daily build on our PPC64LE box here at the UNH-IOL. I did reboot the machine after OFED install. It looks like it is loading fine, it loads without errors:</div><div class="gmail_default" style="font-size:large"><br></div><div class="gmail_default" style="font-size:large">[soesterreich@hyrrokkin:~]$sudo modprobe rpcrdma<br>[soesterreich@hyrrokkin:~]$</div><div class="gmail_default" style="font-size:large"><br></div><div class="gmail_default" style="font-size:large">[soesterreich@hyrrokkin:~]$lsmod | grep rpcrdma<br>rpcrdma 106978 0 <br>rdma_cm 61373 2 rpcrdma,rdma_ucm<br>ib_core 253189 15 iw_cxgb3,iw_cxgb4,rdma_cm,ib_cm,iw_cm,rpcrdma,mlx4_ib,mlx5_ib,ib_ucm,iw_nes,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib<br>compat 13726 20 iw_cxgb3,iw_cxgb4,rdma_cm,cxgb3,cxgb4,ib_cm,iw_cm,rpcrdma,mlx4_en,mlx4_ib,mlx5_ib,ib_ucm,iw_nes,ib_core,ib_umad,ib_uverbs,mlx4_core,mlx5_core,rdma_ucm,ib_ipoib<br>sunrpc 468639 16 nfs,nfsd,rpcsec_gss_krb5,auth_rpcgss,lockd,nfsv4,rpcrdma,nfs_acl<br></div><div class="gmail_default" style="font-size:large"><br></div><div class="gmail_default" style="font-size:large">Looks like he just forgot to reboot. Thank you for your assistance. <br></div><div class="gmail_default" style="font-size:large"><br></div><div class="gmail_default" style="font-size:large">--Stefan <br></div><div class="gmail_default" style="font-size:large"><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 5, 2017 at 4:43 PM, David Z. Dai <span dir="ltr"><<a href="mailto:zdai@linux.vnet.ibm.com" target="_blank">zdai@linux.vnet.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi, Vlad:<br>
No, I didn't reboot the server after a successful OFED installation,<br>
and I didn't restart the openibd either. I don't remember I did this<br>
with OFED-3.18 release.<br>
<br>
I follow your suggestion, did a reboot after the installation. Now it works fine!<br>
[root@rhel73_host1_guest1_<wbr>local_htx ~]# modprobe rpcrdma<br>
[root@rhel73_host1_guest1_<wbr>local_htx ~]#<br>
[root@rhel73_host1_guest1_<wbr>local_htx ~]# lsmod |grep rpcrdma<br>
rpcrdma 106978 0<br>
<br>
[root@rhel73_host1_guest1_<wbr>local_htx ~]# modprobe -r rpcrdma<br>
[root@rhel73_host1_guest1_<wbr>local_htx ~]#<br>
<br>
I haven't tested the functionality yet. At least module is loading fine now.<br>
<br>
Thanks for the help! - David Dai<br>
<div class="HOEnZb"><div class="h5"><br>
On Tue, 2017-09-05 at 23:18 +0300, Vladimir Sokolovsky wrote:<br>
> Hi David,<br>
> Did you reboot the server after OFED installation or reloaded drivers<br>
> with "/etc/init.d/openibd restart" command?<br>
><br>
> Regards,<br>
> Vladimir<br>
><br>
> On 09/05/2017 10:51 PM, David Z. Dai wrote:<br>
> > Hi, Jeff:<br>
> > It has the same error when loading "rpmcrdma" directly.<br>
> > [root@rhel73_host1_guest1_<wbr>local_htx modules]# modprobe rpcrdma<br>
> > modprobe: ERROR: could not insert 'rpcrdma': Exec format error<br>
> ><br>
> > I also change the "rpcrdma.ko" mode from 644 to 744. It still has the<br>
> > same issue.<br>
> ><br>
> > Thanks! - David Dai<br>
> ><br>
> > On Tue, 2017-09-05 at 12:33 -0700, Jeff Becker wrote:<br>
> >> Hi,<br>
> >><br>
> >> I responded to the bugzilla 2633. In short, please try "modprobe<br>
> >> rpcrdma" directly, and just once as it now contains both server and<br>
> >> client. Thanks.<br>
> >><br>
> >> -jeff<br>
> >><br>
> >> On 09/05/2017 12:25 PM, David Z. Dai wrote:<br>
> >>> Hi, Vlad, Jeff:<br>
> >>> Today I got the latest daily build from:<br>
> >>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__downloads.openfabrics.org_OFED_ofed-2D4.8-2D1-2Ddaily_OFED-2D4.8-2D1-2D20170905-2D0757.tgz&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=jE7tivF653Vygp552zF5nqagPKJAuy1x71Qnyf3szIw&m=g18gQ6E3E67ica6Ua-VcSfLqUhqoidoQ6p9JBZX16ZM&s=7P1W9rvFy-ZeD1z4hfqzovK40umK372kBX0EKXyeYfQ&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__<wbr>downloads.openfabrics.org_<wbr>OFED_ofed-2D4.8-2D1-2Ddaily_<wbr>OFED-2D4.8-2D1-2D20170905-<wbr>2D0757.tgz&d=DwICaQ&c=jf_<wbr>iaSHvJObTbx-siA1ZOg&r=<wbr>jE7tivF653Vygp552zF5nqagPKJAuy<wbr>1x71Qnyf3szIw&m=<wbr>g18gQ6E3E67ica6Ua-<wbr>VcSfLqUhqoidoQ6p9JBZX16ZM&s=<wbr>7P1W9rvFy-<wbr>ZeD1z4hfqzovK40umK372kBX0EKXye<wbr>YfQ&e=</a><br>
> >>> This latest daily build includes the NFSoRDMA functionality now.<br>
> >>><br>
> >>> I did a full install on RHEL 7.3 LE OS:<br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# uname -a<br>
> >>> Linux rhel73_host1_guest1_local_htx 3.10.0-514.el7.ppc64le #1 SMP Wed Oct 19<br>
> >>> 11:27:06 EDT 2016 ppc64le ppc64le ppc64le GNU/Linux<br>
> >>><br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# ./<a href="http://install.pl" rel="noreferrer" target="_blank">install.pl</a> --all<br>
> >>><br>
> >>> The installation finished successfully.<br>
> >>><br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# ofed_info -s<br>
> >>> OFED-4.8-1-20170905-0757:<br>
> >>><br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# modinfo xprtrdma<br>
> >>> filename: /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> alias: xprtrdma<br>
> >>> alias: svcrdma<br>
> >>> license: Dual BSD/GPL<br>
> >>> description: RPC/RDMA Transport<br>
> >>> author: Open Grid Computing and Network Appliance, Inc.<br>
> >>> rhelversion: 7.3<br>
> >>> srcversion: 9629610592FC8363A00B592<br>
> >>> depends: sunrpc,compat,ib_core,rdma_cm<br>
> >>> vermagic: 3.10.0-514.el7.ppc64le SMP mod_unload modversions<br>
> >>><br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# modinfo svcrdma<br>
> >>> filename: /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> alias: xprtrdma<br>
> >>> alias: svcrdma<br>
> >>> license: Dual BSD/GPL<br>
> >>> description: RPC/RDMA Transport<br>
> >>> author: Open Grid Computing and Network Appliance, Inc.<br>
> >>> rhelversion: 7.3<br>
> >>> srcversion: 9629610592FC8363A00B592n<br>
> >>> depends: sunrpc,compat,ib_core,rdma_cm<br>
> >>> vermagic: 3.10.0-514.el7.ppc64le SMP mod_unload modversions<br>
> >>><br>
> >>> They are not in "nfsrdma" rpm any more. They are now in "compat-rdma" rpm:<br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# rpm -qf<br>
> >>> /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> compat-rdma-4.8-0.20170905.<wbr>0757.g63258b0.3.10.0_514.el7.<wbr>ppc64le.ppc64le<br>
> >>><br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# rpm -qf<br>
> >>> /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> compat-rdma-4.8-0.20170905.<wbr>0757.g63258b0.3.10.0_514.el7.<wbr>ppc64le.ppc64le<br>
> >>><br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# ls -al /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> -rw-r--r--. 1 root root 6031120 Sep 5 14:24 /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# ls -al /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>> -rw-r--r--. 1 root root 6031120 Sep 5 14:24 /lib/modules/3.10.0-514.el7.<wbr>ppc64le/updates/net/sunrpc/<wbr>xprtrdma/rpcrdma.ko<br>
> >>><br>
> >>> However I see the error during module load time:<br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# modprobe xprtrdma<br>
> >>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error<br>
> >>> [root@rhel73_host1_guest1_<wbr>local_htx OFED-4.8-1-20170905-0757]# modprobe svcrdma<br>
> >>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error<br>
> >>><br>
> >>><br>
> >>> NOTE:<br>
> >>> 1) Both xprtrdma and svcrdma are using same "rpcrdma.ko" (see modinfo output above)<br>
> >>> 2) The mode of "rpcrdma.ko" is 644. Should it be at least 744?<br>
> >>><br>
> >>> Could you please check on this?<br>
> >>><br>
> >>> Thanks! - David<br>
> >>> On Sat, 2017-09-02 at 00:59 +0300, Vladimir Sokolovsky wrote:<br>
> >>>> Applied + build: OFED-4.8-1-20170901-1439<br>
> >>>><br>
> >>>> Regards,<br>
> >>>> Vladimir<br>
> >>>><br>
> >>>> On 09/02/2017 12:44 AM, Jeff Becker wrote:<br>
> >>>>> Hi Vlad,<br>
> >>>>><br>
> >>>>> Please apply the attached patches to compat-rdma.git. Together with<br>
> >>>>> the previous two patches, these have been tested with RHEL7.3 and<br>
> >>>>> SLES12SP2. Thanks.<br>
> >>>>><br>
> >>>>> -jeff<br>
> >>>>><br>
> >>>> ______________________________<wbr>_________________<br>
> >>>> ewg mailing list<br>
> >>>> <a href="mailto:ewg@lists.openfabrics.org">ewg@lists.openfabrics.org</a><br>
> >>>> <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_ewg&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=jE7tivF653Vygp552zF5nqagPKJAuy1x71Qnyf3szIw&m=antYpg37AAv4LKbRqOFw77idqtt-Kxw8KtFSKz_3UJE&s=RNm5Hv7MRR_y-c_JC9pR1ZHCbLAkAtQh5HJCBEOEfj0&e=" rel="noreferrer" target="_blank">https://urldefense.proofpoint.<wbr>com/v2/url?u=http-3A__lists.<wbr>openfabrics.org_mailman_<wbr>listinfo_ewg&d=DwIGaQ&c=jf_<wbr>iaSHvJObTbx-siA1ZOg&r=<wbr>jE7tivF653Vygp552zF5nqagPKJAuy<wbr>1x71Qnyf3szIw&m=<wbr>antYpg37AAv4LKbRqOFw77idqtt-<wbr>Kxw8KtFSKz_3UJE&s=RNm5Hv7MRR_<wbr>y-c_<wbr>JC9pR1ZHCbLAkAtQh5HJCBEOEfj0&<wbr>e=</a><br>
> ><br>
><br>
<br>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><b>-----------------------------------------------<br>Cheers,<br>Stefan Oesterreich</b><div><b>High Performance Computing</b></div><div><b>UNH InterOperability Laboratory<br>------------------------------------------------<br></b></div></div></div></div></div></div></div></div></div>
</div>