[ofa-general] Re: Installing OFED 1.4.1 in XenServer

Ross Smith myxiplx at googlemail.com
Thu Jun 25 03:41:26 PDT 2009 

Ok, Infiniband is working fine with XenServer, but I'm getting poor
performance from NFS so want to try NFS-RDMA.

It doesn't look like the default install has included this, so I could
do with some help getting this running.

My first attempt was to run install.pl again, and choose all modules.
That needed some more prerequisites,  so I installed pciutils-devel,
tcl-devel, tk and tcsh.  However building all with install.pl now
fails with the error:

"Failed to build ofa_kernel RPM
See /tmp/OFED.10263.logs/ofa_kernel.rpmbuild.log"

I then tried a custom build, but no matter what I tried, it didn't
seem to register that I had chosen nfs-rdma (I'm guessing though that
it's included in kernel-ib and just isn't shown on the confirmation
screen).  However, the custom build also ended with that same error,
even after I removed the extre prerequisites.

I eventually did get it to build, but I don't remember which pieces I
picked from the custom build, and there are so many I don't actually
know which I do or don't need.  I also now can't work out which .rpm
file it created for nfsrdma.

Can anybody help me understand what I need to do to get the .rpm for
nfsrdma so I can install it on the live server for testing?  All I
want are the default modules plus nfsrdma.

thanks,

Ross

PS.  In case it helps, the last 20 lines of that error log are:

# tail /tmp/OFED.10263.logs/ofa_kernel.rpmbuild.log -n 20
                 from
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband/ulp/qlgc_vnic/vnic_main.c:35:
include/linux/sysctl.h:58:1: warning: this is the location of the
previous definition
In file included from
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband/ulp/qlgc_vnic/vnic_viport.h:38,
                 from
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband/ulp/qlgc_vnic/vnic_main.c:46:
/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband/ulp/qlgc_vnic/vnic_multicast.h:49:
error: redefinition of 'struct mc_info'
make[4]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband/ulp/qlgc_vnic/vnic_main.o]
Error 1
make[3]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband/ulp/qlgc_vnic]
Error 2
make[2]: *** [/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1/drivers/infiniband]
Error 2
make[1]: *** [_module_/var/tmp/OFED_topdir/BUILD/ofa_kernel-1.4.1] Error 2
make[1]: Leaving directory
`/usr/src/kernels/2.6.18-128.1.6.el5.xs5.5.0.496.1012xen-i686'
make: *** [kernel] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.6235 (%build)


RPM build errors:
    user vlad does not exist - using root
    group vlad does not exist - using root
    user vlad does not exist - using root
    group vlad does not exist - using root
    Bad exit status from /var/tmp/rpm-tmp.6235 (%build)







On Wed, Jun 24, 2009 at 6:09 PM, Ross Smith<myxiplx at googlemail.com> wrote:
> Me again,
>
> Well, the installation went pretty well but there was an error during
> the installation with iscsi:
> "Failed to install iscsi-initiator-utils RPM
> See /tmp/OFED.6760.logs/iscsi-initiator-utils.rpminstall.log "
>
> Removing open-iscsi seemed to get around this, and the version
> included with OFED is slighly newer, so fingers crossed it won't cause
> any problems:
> # yum remove open-iscsi
>
> Tomorrow I'll see if it plays nicely with Xen.
>
> Ross
>
>
> On Wed, Jun 24, 2009 at 4:41 PM, Ross Smith<myxiplx at googlemail.com> wrote:
>> Thanks Vladimir, that sounds very neat and tidy :-)
>>
>> I'll be sure to let the list know how my testing goes with XenServer.
>>
>>
>>
>> On Wed, Jun 24, 2009 at 4:30 PM, Vladimir
>> Sokolovsky<vlad at dev.mellanox.co.il> wrote:
>>> Ross Smith wrote:
>>>>
>>>> Replying to my own email since I've made progress and now have a new
>>>> question.
>>>>
>>>> OFED 1.4.1 compiles and installs find under the XenServer 5.5 DDK.  My
>>>> question now is how do I transfer these drivers to a live server?
>>>>
>>>> Are there details anywhere about what files need transferring, and how
>>>> they should be loaded?
>>>>
>>>> Alternatively, do I need to attempt to transfer the relevant kernel
>>>> source files from the XenServer DDK to a live server?
>>>>
>>>> thanks,
>>>>
>>>> Ross
>>>>
>>>
>>> Hi Ross,
>>> After installing OFED-1.4.1 you will have binary RPMs under
>>> OFED-1.4.1/RPMS/... directory.
>>> So, to install OFED-1.4.1 on the live server you should run the
>>> same install command (from the same OFED-1.4.1 directory;
>>> you can copy it to the live server if it is not accessible through NFS)
>>> on your server and it will install binary RPMs that were previously
>>> created.
>>> It will not require kernel sources on the live server.
>>>
>>> Regards,
>>> Vladimir
>>>
>>>>
>>>>
>>>> On Wed, Jun 24, 2009 at 12:21 PM, Ross Smith<myxiplx at googlemail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> Hi guys,
>>>>>
>>>>> I'm very new to Linux, and trying to install OFED in XenServer, is
>>>>> anybody able to help out?
>>>>>
>>>>> XenServer is based on CentOS 5.2, and I'm informed that it should be
>>>>> possible to install this version of OFED, but I'm struggling to get
>>>>> the installation to compile.
>>>>>
>>>>> I believe the proper way to compile the drivers is to use the Citrix
>>>>> Driver Development Kit, which includes all the source for XenServer.
>>>>> However that's supplied as a 2GB virtual machine, and runs out of disk
>>>>> space if I try to install the OFED prerequisits.
>>>>>
>>>>> So instead I'm trying to compile it directly on Dom0 of a XenServer.
>>>>> The prerequisits have installed fine, but the install.pl script fails
>>>>> with:
>>>>>
>>>>> "/lib/modules/2.6.18-92.1.10.el5.xs5.0.0.426.647xen/build/scripts is
>>>>> required to build kernel-ib RPM.
>>>>> Please install the corresponding kernel-source or kernel-devel RPM."
>>>>>
>>>>> On the DDK, I can see that those folders all exist, but I can't find
>>>>> how to install those on a live server.  I've found
>>>>> kernel-2.6.18.92.1.10.el5.xs5.0.0.426.647.src.rpm on the DDK source
>>>>> disk, but installing that does not help.
>>>>>
>>>>> If I were to simply copy the files from the DDK to the live server,
>>>>> could somebody advise me on what I would need to copy.  Do I need
>>>>> anything outside of the /lib/modules/... directory?
>>>>>
>>>>> thanks,
>>>>>
>>>>> Ross
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> general mailing list
>>>> general at lists.openfabrics.org
>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>>
>>>> To unsubscribe, please visit
>>>> http://openib.org/mailman/listinfo/openib-general
>>>>
>>>>
>>>
>>>
>>
>

More information about the general mailing list