[ewg] OFED 1.2 RC2 on rhel4u4 x86_64

Steffen Persvold steffen.persvold at scali.com
Fri May 4 14:29:34 PDT 2007 

Etta,
 
Of course my system has the /usr/lib64 directory. Using install or build doesn't seem to make a difference, the problems seems to be that when the 64bit libraries are compiled and installed they're installed in <RPM build path>/usr/lib and not <RPM build path>/usr/lib64 and thus when rpmbuild gets to compiling and installing the 32bit libraries the 64bit libraries are overwritten... I don't know too much about the Make files and configure scripts inside the .src.rpm files to understand exactly why it tells it to install the 64bit libraries in /usr/lib and not in /usr/lib64...
 
Anyone have any insight on that ??
 
Cheers,
 
Steffen Persvold
Technical Director Americas
tel. 508-281-7100 x401
fax. 508-281-7171

http://www.scali.com/
Scaling the Linux datacenter

________________________________

From: Chieng Etta [mailto:etta at systemfabricworks.com]
Sent: Fri 5/4/2007 10:28 AM
To: Steffen Persvold; vlad at dev.mellanox.co.il
Cc: openfabrics-ewg at openib.org; openib-general at openib.org
Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64



Steffen,

 

The installation should be the same on either ES or AS.   

I assume that your system should have /usr/lib64 directory.  Would you be able to install rc2 by using ./install.sh script?

 

Thanks,

Etta

 

________________________________

From: Steffen Persvold [mailto:steffen.persvold at scali.com] 
Sent: Thursday, May 03, 2007 5:38 PM
To: Chieng Etta; vlad at dev.mellanox.co.il
Cc: openfabrics-ewg at openib.org; openib-general at openib.org
Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64

 

So I don't understand it then... Why are my RPMs only containing one of the two versions. I'm running on ES and not AS but that shouldn't really matter...

 

This output that you list :

 

[root at sfw1 redhat-release-4AS-5.5]# rpm -qpl libibverbs-1.1-0.x86_64.rpm
/etc/ld.so.conf.d/ofed.conf
/usr/lib/libibverbs.so.1
/usr/lib/libibverbs.so.1.0.0
/usr/lib64/libibverbs.so.1
/usr/lib64/libibverbs.so.1.0.0

Is exactly what I would have expected as well, but my RPM says :

 

[root at pe1850-1 redhat-release-4ES-5.5]# pwd
/root/OFED-1.2-rc2/RPMS/redhat-release-4ES-5.5
[root at pe1850-1 redhat-release-4ES-5.5]# rpm -qpl libibverbs-1.1-0.x86_64.rpm
/etc/ld.so.conf.d/ofed.conf
/usr/lib/libibverbs.so.1
/usr/lib/libibverbs.so.1.0.0

I'm lookin through the build log (/tmp/OFED.build.xxx.log) and both versions get compiled, but it looks like the 32bit libraries (which gets compiled last) overwrites the 64bit libraries in the "make install" section because both ends up in /usr/lib :

 

(64bit section of the build) :

 

/usr/bin/install -c src/.libs/libibverbs.so.1.0.0 /var/tmp/OFED/usr/lib/libibverbs.so.1.0.0
(cd /var/tmp/OFED/usr/lib && { ln -s -f libibverbs.so.1.0.0 libibverbs.so.1 || { rm -f libibverbs.so.1 && ln -s libibverbs.so.1.0.0 libibverbs.so.1; }; })
(cd /var/tmp/OFED/usr/lib && { ln -s -f libibverbs.so.1.0.0 libibverbs.so || { rm -f libibverbs.so && ln -s libibverbs.so.1.0.0 libibverbs.so; }; })

 

(32bit section of the build) :

/usr/bin/install -c src/.libs/libibverbs.so.1.0.0 /var/tmp/OFED/usr/lib/libibverbs.so.1.0.0
(cd /var/tmp/OFED/usr/lib && { ln -s -f libibverbs.so.1.0.0 libibverbs.so.1 || { rm -f libibverbs.so.1 && ln -s libibverbs.so.1.0.0 libibverbs.so.1; }; })
(cd /var/tmp/OFED/usr/lib && { ln -s -f libibverbs.so.1.0.0 libibverbs.so || { rm -f libibverbs.so && ln -s libibverbs.so.1.0.0 libibverbs.so; }; })

 

So the question is, why is the 64bit section ending up in <buildpath>/usr/lib in the first place ???

 

I do see this though :

 

/bin/rm -f /var/tmp/OFEDRPM/BUILD/ofa_user-1.2/configure.cache
cd /var/tmp/OFEDRPM/BUILD/ofa_user-1.2/src/userspace/libibverbs
Running: env ac_cv_lib_ibverbs_ibv_get_device_list=yes ac_cv_header_infiniband_driver_h=yes ac_cv_func_ibv_read_sysfs_file=yes ac_cv_func_ibv_dontfork_range=
yes ac_cv_func_ibv_dofork_range=yes ac_cv_func_ibv_register_driver=yes HAVE_IBV_DEVICE_LIBRARY_EXTENSION_TRUE=yes  ./configure --cache-file=/var/tmp/OFEDRPM/
BUILD/ofa_user-1.2/configure.cache --disable-libcheck --prefix /usr --libdir /usr/lib --mandir=/usr/man --sysconfdir=/usr/etc CPPFLAGS="-I../libibverbs/include"

 

--libdir /usr/lib ??? shouldn't that be --libdir /usr/lib64 for the 64bit section ?

 

Cheers,

Steffen Persvold
Technical Director Americas
tel. 508-281-7100 x401
fax. 508-281-7171
 
http://www.scali.com/
Scaling the Linux datacenter

 

________________________________

From: Chieng Etta [mailto:etta at systemfabricworks.com]
Sent: Thu 5/3/2007 3:26 PM
To: Steffen Persvold; vlad at dev.mellanox.co.il
Cc: openfabrics-ewg at openib.org; openib-general at openib.org
Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64

Hi Steffen,

After removing all the OFED packages by using ./uninstall.sh, I tried
./build.sh to build the RPMs then installed libibverbs-1.1-0.x86_64.rpm onto
system.  "libibverbs.so.1.0.0" was installed under the right directories
(/usr/lib and /usr/lib64).  Please see the output below. 
Thanks,
Etta

[root at sfw1 etc]# cat /etc/*release
Red Hat Enterprise Linux AS release 4 (Nahant Update 4)
[root at sfw1 etc]# uname -a
Linux sfw1.sfw.int 2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:32:02 EDT 2006 x86_64
x86_64 x86_64 GNU/Linux

[root at sfw1 lib64]# pwd
/usr/lib64
[root at sfw1 lib64]# ll libibverbs*
ls: libibverbs*: No such file or directory

[root at sfw1 lib64]# rpm -aq |grep libibverbs

[root at sfw1 lib64]# cd -
/root/images/OFED-1.2-rc2/RPMS/redhat-release-4AS-5.5
[root at sfw1 redhat-release-4AS-5.5]# rpm -qpl libibverbs-1.1-0.x86_64.rpm
/etc/ld.so.conf.d/ofed.conf
/usr/lib/libibverbs.so.1
/usr/lib/libibverbs.so.1.0.0
/usr/lib64/libibverbs.so.1
/usr/lib64/libibverbs.so.1.0.0

[root at sfw1 redhat-release-4AS-5.5]# rpm -ivh libibverbs-1.1-0.x86_64.rpm
Preparing...             ########################################### [100%]
   1:libibverbs          ########################################### [100%]

[root at sfw1 redhat-release-4AS-5.5]# rpm -qp --qf "%{arch}\n"
libibverbs-1.1-0.x86_64.rpm
x86_64

[root at sfw1 redhat-release-4AS-5.5]# cd -
/usr/lib64
[root at sfw1 lib64]# rpm -aq |grep libibverbs
libibverbs-1.1-0

[root at sfw1 lib64]# ll libibverbs*
lrwxrwxrwx  1 root root     19 May  3 13:50 libibverbs.so.1 ->
libibverbs.so.1.0.0
-rwxr-xr-x  1 root root 200993 May  3 13:18 libibverbs.so.1.0.0

[root at sfw1 lib64]# file libibverbs.so.1.0.0
libibverbs.so.1.0.0: ELF 64-bit LSB shared object, AMD x86-64, version 1
(SYSV), not stripped

[root at sfw1 lib]# cd /usr/lib
[root at sfw1 lib]# file libibverbs.so.1.0.0
libibverbs.so.1.0.0: ELF 32-bit LSB shared object, Intel 80386, version 1
(SYSV), not stripped

[root at sfw1 etc]# cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/ofed/lib64

[root at sfw1 etc]# cat /etc/ld.so.conf.d/ofed.conf
/usr/lib64
/usr/lib
   

-----Original Message-----
From: ewg-bounces at lists.openfabrics.org
[mailto:ewg-bounces at lists.openfabrics.org] On Behalf Of Steffen Persvold
Sent: Thursday, May 03, 2007 10:26 AM
To: vlad at dev.mellanox.co.il
Cc: openfabrics-ewg at openib.org; openib-general at openib.org
Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64

Vladimir,

Nope. Still the same issue. The RPMs will only contain one set of
libraries and it is always in /usr/lib (if I set the build_32bit=0
option I get the 64bit libraries but in the wrong directory).

Seriously, am I the only one seeing this ? I would think rhel4 u4 was a
very normal test platform ?

Cheers,

Steffen Persvold
Technical Director Americas
tel. 508-281-7100 x401
fax. 508-281-7171

http://www.scali.com/
Scaling the Linux datacenter


> -----Original Message-----
> From: Vladimir Sokolovsky [mailto:vlad at dev.mellanox.co.il]
> Sent: Thursday, May 03, 2007 9:07 AM
> To: Steffen Persvold
> Cc: openfabrics-ewg at openib.org; openib-general at openib.org
> Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64
>
> Please see if this happens in OFED-1.2-20070503-0600.
> But first uninstall the previous OFED version with ofed_uninstall.sh
> command.
>
> Thanks,
>
> Regards,
> Vladimir
>
> On Wed, 2007-05-02 at 11:30 -0400, Steffen Persvold wrote:
> > Hmm,
> >
> > so I tried something. I put :
> >
> > build_32bit=0
> >
> > into my ofed.conf file and rebuilt (build.sh -c ofed.conf). This
time
> > it built 64bit libraries, but it puts them in the wrong directory :
> >
> > # rpm -qpl ../libibverbs-1.1-0.x86_64.rpm
> > /etc/ld.so.conf.d/ofed.conf
> > /usr/lib/libibverbs.so.1
> > /usr/lib/libibverbs.so.1.0.0
> >
> > # file /usr/lib/libibverbs.so.1.0.0
> > /usr/lib/libibverbs.so.1.0.0: ELF 64-bit LSB shared object, AMD
> > x86-64, version 1 (SYSV), not stripped
> >
> > So what's up ??
> >
> > Cheers,
> > Steffen Persvold
> > Technical Director Americas
> > tel. 508-281-7100 x401
> > fax. 508-281-7171
> >
> > http://www.scali.com/
> > Scaling the Linux datacenter
> >
> >
> >
______________________________________________________________________
> > From: Steffen Persvold
> > Sent: Wed 5/2/2007 10:30 AM
> > To: Steffen Persvold; Vladimir Sokolovsky
> > Cc: openfabrics-ewg at openib.org; openib-general at openib.org
> > Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64
> >
> >
> > Also,
> >
> > If I look at the /etc/ld.so.conf/ofed.conf file I have :
> >
> > # cat ofed.conf
> > /usr/lib
> > /usr/lib
> >
> >
> > which seems kinda weird ? :)
> >
> > Cheers,
> >
> > Steffen Persvold
> > Technical Director Americas
> > tel. 508-281-7100 x401
> > fax. 508-281-7171
> >
> > http://www.scali.com/
> > Scaling the Linux datacenter
> >
> >
> >
______________________________________________________________________
> > From: ewg-bounces at lists.openfabrics.org on behalf of Steffen
Persvold
> > Sent: Wed 5/2/2007 10:20 AM
> > To: Vladimir Sokolovsky
> > Cc: openfabrics-ewg at openib.org; openib-general at openib.org
> > Subject: RE: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64
> >
> >
> > Nope :
> >
> >
> > [redhat-release-4ES-5.5]# rpm -qpl libibverbs-1.1-0.x86_64.rpm
> > /etc/ld.so.conf.d/ofed.conf
> > /usr/lib/libibverbs.so.1
> > /usr/lib/libibverbs.so.1.0.0
> > [redhat-release-4ES-5.5]#
> >
> > So the RPM got built, but without 64bit libraries. Now if it was the
> > other way around (i.e no 32bit libraries) I could have understood it
> > (as 32bit is an option on x86_64), but not having the native 64bit
> > libraries is not so easy to understand :)
> >
> > cheers,
> > Steffen Persvold
> > Technical Director Americas
> > tel. 508-281-7100 x401
> > fax. 508-281-7171
> >
> > http://www.scali.com/
> > Scaling the Linux datacenter
> >
> >
> >
______________________________________________________________________
> > From: ewg-bounces at lists.openfabrics.org on behalf of Vladimir
> > Sokolovsky
> > Sent: Wed 5/2/2007 10:05 AM
> > To: Steffen Persvold
> > Cc: openfabrics-ewg at openib.org; openib-general at openib.org
> > Subject: Re: [ewg] OFED 1.2 RC2 on rhel4u4 x86_64
> >
> >
> > Don't you have /usr/lib64/libibverbs.so.1.0.0?
> >
> > Regards,
> > Vladimir
> >
> > On Wed, 2007-05-02 at 10:00 -0400, Steffen Persvold wrote:
> > > Folks,
> > >
> > > I used the build.sh script to build the above mentioned packages
on
> > > rhel4u4 x86_64, but for some reason it only compiles 32bit
libraries
> > > (even if the packages are named x86_64) :
> > >
> > > # rpm -qp --qf "%{arch}\n" libibverbs-1.1-0.x86_64.rpm
> > > x86_64
> > >
> > > (after installing it) :
> > >
> > > # file /usr/lib/libibverbs.so.1.0.0
> > > /usr/lib/libibverbs.so.1.0.0: ELF 32-bit LSB shared object, Intel
> > > 80386, version 1 (SYSV), not stripped
> > >
> > > What did I do wrong ??
> > >
> > > Cheers,
> > > Steffen Persvold
> > > Technical Director Americas
> > > tel. 508-281-7100 x401
> > > fax. 508-281-7171
> > >
> > > http://www.scali.com/
> > > Scaling the Linux datacenter
> > > _______________________________________________
> > > ewg mailing list
> > > ewg at lists.openfabrics.org
> > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
> >
> > _______________________________________________
> > ewg mailing list
> > ewg at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
> >
> >
> >

_______________________________________________
ewg mailing list
ewg at lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20070504/9772fca4/attachment.html>

More information about the ewg mailing list