[ewg] NFS/RDMA backport for OFED 4.8-1 part 3

Tue Sep 5 13:48:34 PDT 2017

Hi David,
Yes, you have to reboot or reload drivers as you tried to load the new 
rpcrdma kernel module over the old version (in-box or previous OFED 
build) of ib_core and not the one coming with the same OFED.

Regards,
Vladimir

On 09/05/2017 11:43 PM, David Z. Dai wrote:
> Hi, Vlad:
>    No, I didn't reboot the server after a successful OFED installation,
> and I didn't restart the openibd either. I don't remember I did this
> with OFED-3.18 release.
>
> I follow your suggestion, did a reboot after the installation. Now it works fine!
> [root at rhel73_host1_guest1_local_htx ~]# modprobe rpcrdma
> [root at rhel73_host1_guest1_local_htx ~]#
> [root at rhel73_host1_guest1_local_htx ~]# lsmod |grep rpcrdma
> rpcrdma               106978  0
>
> [root at rhel73_host1_guest1_local_htx ~]# modprobe -r rpcrdma
> [root at rhel73_host1_guest1_local_htx ~]#
>
> I haven't tested the functionality yet. At least module is loading fine now.
>
> Thanks for the help! - David Dai
>
> On Tue, 2017-09-05 at 23:18 +0300, Vladimir Sokolovsky wrote:
>> Hi David,
>> Did you reboot the server after OFED installation or reloaded drivers
>> with "/etc/init.d/openibd restart" command?
>>
>> Regards,
>> Vladimir
>>
>> On 09/05/2017 10:51 PM, David Z. Dai wrote:
>>> Hi, Jeff:
>>> It has the same error when loading "rpmcrdma" directly.
>>> [root at rhel73_host1_guest1_local_htx modules]# modprobe rpcrdma
>>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error
>>>
>>> I also change the "rpcrdma.ko" mode from 644 to 744. It still has the
>>> same issue.
>>>
>>> Thanks! - David Dai
>>>
>>> On Tue, 2017-09-05 at 12:33 -0700, Jeff Becker wrote:
>>>> Hi,
>>>>
>>>> I responded to the bugzilla 2633. In short, please try "modprobe
>>>> rpcrdma" directly, and just once as it now contains both server and
>>>> client. Thanks.
>>>>
>>>> -jeff
>>>>
>>>> On 09/05/2017 12:25 PM, David Z. Dai wrote:
>>>>> Hi, Vlad, Jeff:
>>>>> Today I got the latest daily build from:
>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__downloads.openfabrics.org_OFED_ofed-2D4.8-2D1-2Ddaily_OFED-2D4.8-2D1-2D20170905-2D0757.tgz&d=DwICaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=jE7tivF653Vygp552zF5nqagPKJAuy1x71Qnyf3szIw&m=g18gQ6E3E67ica6Ua-VcSfLqUhqoidoQ6p9JBZX16ZM&s=7P1W9rvFy-ZeD1z4hfqzovK40umK372kBX0EKXyeYfQ&e=
>>>>> This latest daily build includes the NFSoRDMA functionality now.
>>>>>
>>>>> I did a full install on RHEL 7.3 LE OS:
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# uname -a
>>>>> Linux rhel73_host1_guest1_local_htx 3.10.0-514.el7.ppc64le #1 SMP Wed Oct 19
>>>>> 11:27:06 EDT 2016 ppc64le ppc64le ppc64le GNU/Linux
>>>>>
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ./install.pl  --all
>>>>>
>>>>> The installation finished successfully.
>>>>>
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ofed_info -s
>>>>> OFED-4.8-1-20170905-0757:
>>>>>
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# modinfo xprtrdma
>>>>> filename:      /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> alias:          xprtrdma
>>>>> alias:          svcrdma
>>>>> license:        Dual BSD/GPL
>>>>> description:    RPC/RDMA Transport
>>>>> author:         Open Grid Computing and Network Appliance, Inc.
>>>>> rhelversion:    7.3
>>>>> srcversion:     9629610592FC8363A00B592
>>>>> depends:        sunrpc,compat,ib_core,rdma_cm
>>>>> vermagic:       3.10.0-514.el7.ppc64le SMP mod_unload modversions
>>>>>
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# modinfo svcrdma
>>>>> filename:      /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> alias:          xprtrdma
>>>>> alias:          svcrdma
>>>>> license:        Dual BSD/GPL
>>>>> description:    RPC/RDMA Transport
>>>>> author:         Open Grid Computing and Network Appliance, Inc.
>>>>> rhelversion:    7.3
>>>>> srcversion:     9629610592FC8363A00B592n
>>>>> depends:        sunrpc,compat,ib_core,rdma_cm
>>>>> vermagic:       3.10.0-514.el7.ppc64le SMP mod_unload modversions
>>>>>
>>>>> They are not in "nfsrdma" rpm any more. They are now in "compat-rdma" rpm:
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# rpm -qf
>>>>> /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> compat-rdma-4.8-0.20170905.0757.g63258b0.3.10.0_514.el7.ppc64le.ppc64le
>>>>>
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# rpm -qf
>>>>> /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> compat-rdma-4.8-0.20170905.0757.g63258b0.3.10.0_514.el7.ppc64le.ppc64le
>>>>>
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ls -al /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> -rw-r--r--. 1 root root 6031120 Sep  5 14:24 /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# ls -al /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>> -rw-r--r--. 1 root root 6031120 Sep  5 14:24 /lib/modules/3.10.0-514.el7.ppc64le/updates/net/sunrpc/xprtrdma/rpcrdma.ko
>>>>>
>>>>> However I see the error during module load time:
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# modprobe xprtrdma
>>>>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error
>>>>> [root at rhel73_host1_guest1_local_htx OFED-4.8-1-20170905-0757]# modprobe svcrdma
>>>>> modprobe: ERROR: could not insert 'rpcrdma': Exec format error
>>>>>
>>>>>
>>>>> NOTE:
>>>>> 1) Both xprtrdma and svcrdma are using same "rpcrdma.ko"  (see modinfo output above)
>>>>> 2) The mode of "rpcrdma.ko" is 644. Should it be at least 744?
>>>>>
>>>>> Could you please check on this?
>>>>>
>>>>> Thanks! - David
>>>>> On Sat, 2017-09-02 at 00:59 +0300, Vladimir Sokolovsky wrote:
>>>>>> Applied + build: OFED-4.8-1-20170901-1439
>>>>>>
>>>>>> Regards,
>>>>>> Vladimir
>>>>>>
>>>>>> On 09/02/2017 12:44 AM, Jeff Becker wrote:
>>>>>>> Hi Vlad,
>>>>>>>
>>>>>>> Please apply the attached patches to compat-rdma.git. Together with
>>>>>>> the previous two patches, these have been tested with RHEL7.3 and
>>>>>>> SLES12SP2. Thanks.
>>>>>>>
>>>>>>> -jeff
>>>>>>>
>>>>>> _______________________________________________
>>>>>> ewg mailing list
>>>>>> ewg at lists.openfabrics.org
>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_ewg&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=jE7tivF653Vygp552zF5nqagPKJAuy1x71Qnyf3szIw&m=antYpg37AAv4LKbRqOFw77idqtt-Kxw8KtFSKz_3UJE&s=RNm5Hv7MRR_y-c_JC9pR1ZHCbLAkAtQh5HJCBEOEfj0&e=
>