[ofa-general] Re: Infiniband packages for Hardy

[Christopher Tanner]

Thanks for the info Roland. I just subscribed to the openfabrics list,  
so I'll start emailing there from now on.

I created your script and the mlx4_ib module is loading, so that's a  
good step. However, OpenMPI is still not finding the HCAs. I'm getting  
another error (on each node) that I didn't post in the Ubuntu forum:

libibverbs: Fatal: couldn't read uverbs ABI version.

I uninstalled and reinstalled the libibverbs1 package (apt-get purge,  
apt-get install), but the error persists. Attached is an output from  
an lsmod (just to confirm the loaded modules).
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: lsmod_out.txt
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080826/cbd1522d/attachment.txt>
-------------- next part --------------



-------------------------------------------
Chris Tanner
Space Systems Design Lab
Georgia Institute of Technology
christopher.tanner at gatech.edu
-------------------------------------------



On Aug 26, 2008, at 11:11 AM, Roland Dreier wrote:

>> Apologies for contacting you directly - are you available to provide
>> some support for the Infiniband packages you built for Ubuntu 8.04? I
>> posted my issue on the Ubuntu forums a while ago
>> (http://ubuntuforums.org/showthread.php?t=896924 ), but haven't
>> received any response yet.
>
> No problem, I don't really read web forums.  The best way to get help
> with IB in general is to email the list general at lists.openfabrics.org.
>
> In your particular case I would guess the problem is that you don't  
> have
> the mlx4_ib module loaded; by default only the mlx4_core module will  
> be
> auto-loaded.  To test this, you can do "sudo modprobe mlx4_ib" by hand
> and try it.  (This is all based on the fact that you installed the
> libmlx4 packages, so I'm assuming you have ConnectX cards).
>
> A better solution would be to create a file named
> /etc/modprobe.d/mlx4_core with the line
>
> install mlx4_core /sbin/modprobe --ignore-install mlx4_core && /sbin/ 
> modprobe mlx4_ib
>
> in it, which should make mlx4_ib load by default on boot.
>
>> Also, if you are willing to provide support, I would like your
>> permission to post some of your suggestions and the ultimate
>> resolution on the Ubuntu forum so that others can benefit from our
>> dialogue.
>
> Go ahead and add it to the thread.
>
> - R.