[ofa-general] Permission denied

[Christopher Tanner]

I'm receiving this error when I try to execute a mpi executable:

[node2][0,1,1][btl_openib_component.c:466:init_one_hca] error  
obtaining device context for mthca0 errno says Permission denied

--------------------------------------------------------------------------
WARNING: There were errors during IB HCA initialization on host 'node2'.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
WARNING: There is at least on IB HCA found on host 'node2', but there is
no active ports detected. This is most certainly not what you wanted.
Check your cables and SM configuration.
--------------------------------------------------------------------------

I'm confused about the 'Permission denied'. My user is part of the  
group 'rdma', which I thought was supposed to give them permission to  
access the Infiniband devices. I'm also confused because the trivial  
test cases such as 'Hello World' and 'hostname' execute on all nodes  
without errors.

The 'no active ports' is also curious. On the master node, I am  
running OpenSM and it indicates that the port is active (using  
ibv_devinfo). However, I notice that the 'ibv_devinfo' command can  
only be run by root. Is this an indication that permissions are not  
set correctly?

As another note, my cluster is running Ubuntu 8.04, so I couldn't use  
the OFED scripts to install the Infiniband drivers, so I had to  
compile and install everything from source (which seemed to go fine).

Thanks for your help!

-------------------------------------------
Chris Tanner
Space Systems Design Lab
Georgia Institute of Technology
christopher.tanner at gatech.edu
-------------------------------------------