[ofa-general] Installing openIB on Linux FC5

[Hossein Pourreza]

On Wed, Jun 06, 2007 at 10:21:31AM -0400, Hal Rosenstock wrote:
> On Wed, 2007-06-06 at 10:08, Hossein Pourreza wrote:
> > Hi,
> > 
> > Many thanks for your reply. I really appreciate that.
> > 
> > Our cluster uses Mellanox Technologies MT23108 InfiniHost (rev a1) and Sun 9P switch. 
> > Out switch has its own SubnetManager and whenever I try to run opensm, I get an error 
> > saying that there is another sm running with a mismatch key.
> > 
> > The result of running ibstat is like this:
> > 
> > 		CA type: MT23108
> >         Number of ports: 2
> >         Firmware version: 3.3.2
> >         Hardware version: a1
> >         Node GUID: 0x0003ba0001001788
> >         System image GUID: 0x0003ba000100178b
> >         Port 1:
> >                 State: Active
> >                 Physical state: LinkUp
> >                 Rate: 10
> >                 Base lid: 2
> >                 LMC: 0
> >                 SM lid: 1
> >                 Capability mask: 0x00510a68
> >                 Port GUID: 0x0003ba0001001789
> > 		Port 2:
> >                 State: Down
> >                 Physical state: Polling
> >                 Rate: 2
> >                 Base lid: 0
> >                 LMC: 0
> >                 SM lid: 0
> >                 Capability mask: 0x00510a68
> >                 Port GUID: 0x0003ba000100178a
> > 
> > Is there anything wrong with this output?
> 
> Nothing wrong with the output :-) but is your port connected ? It
> appears there is some connectivity problem as Physical state is not
> LinkUp (and hence State is  Down) so SM cannot configure it.

I only use port 1 of each HCA and I just connected those to the switch. Should I
connect both ports? There are only 9 ports available on our switch and we have 5
nodes (10 ports in total).

Thanks again for all you help
Hossein


> 
> -- Hal
> 
> > Many thanks for your kind help
> > Hossein
> > On Wed, Jun 06, 2007 at 09:09:53AM +0300, Tziporet Koren wrote:
> > > Hossein Pourreza wrote:
> > > >Hi all,
> > > >
> > > >I am new to infiniband stuff and am trying to configure an infiniband-based
> > > >cluster using Linux FC 5. I downloaded the OFED-1.0 and tried to install 
> > > >it on
> > > >cluster nodes. Now I can load the kernel modules without any error but I 
> > > >cannot
> > > >run a simple test like ibv_ud_pingpong to check the connectivity of nodes 
> > > >in
> > > >user-level.
> > > >
> > > >
> > > >  
> > > Have you run opensm?
> > > You can run ibstat on each node to see ports are active
> > > 
> > > Tziporet

-- 
Hossein Pourreza		 			mail:<pourreza at cs.umanitoba.ca>    
Department of Computer Science		URL: http://www.cs.umanitoba.ca/~pourreza
University of Manitoba  			Phone: 204-488-5611            
Winnipeg, Manitoba, Canada R3T 2N2