[ofa-general] Installing openIB on Linux FC5

[Makia Minich]

I think that Hal missed that Port 1 is in active/link up state.

More importantly, are you looking to replace your internal SubnetManager and 
just use OpenSM?  If so, you'll need to go into the switch and disable it, 
then bring up opensm.

On Wednesday 06 June 2007 10:45:57 am Hossein Pourreza wrote:
> On Wed, Jun 06, 2007 at 10:21:31AM -0400, Hal Rosenstock wrote:
> > On Wed, 2007-06-06 at 10:08, Hossein Pourreza wrote:
> > > Hi,
> > >
> > > Many thanks for your reply. I really appreciate that.
> > >
> > > Our cluster uses Mellanox Technologies MT23108 InfiniHost (rev a1) and
> > > Sun 9P switch. Out switch has its own SubnetManager and whenever I try
> > > to run opensm, I get an error saying that there is another sm running
> > > with a mismatch key.
> > >
> > > The result of running ibstat is like this:
> > >
> > > 		CA type: MT23108
> > >         Number of ports: 2
> > >         Firmware version: 3.3.2
> > >         Hardware version: a1
> > >         Node GUID: 0x0003ba0001001788
> > >         System image GUID: 0x0003ba000100178b
> > >         Port 1:
> > >                 State: Active
> > >                 Physical state: LinkUp
> > >                 Rate: 10
> > >                 Base lid: 2
> > >                 LMC: 0
> > >                 SM lid: 1
> > >                 Capability mask: 0x00510a68
> > >                 Port GUID: 0x0003ba0001001789
> > > 		Port 2:
> > >                 State: Down
> > >                 Physical state: Polling
> > >                 Rate: 2
> > >                 Base lid: 0
> > >                 LMC: 0
> > >                 SM lid: 0
> > >                 Capability mask: 0x00510a68
> > >                 Port GUID: 0x0003ba000100178a
> > >
> > > Is there anything wrong with this output?
> >
> > Nothing wrong with the output :-) but is your port connected ? It
> > appears there is some connectivity problem as Physical state is not
> > LinkUp (and hence State is  Down) so SM cannot configure it.
>
> I only use port 1 of each HCA and I just connected those to the switch.
> Should I connect both ports? There are only 9 ports available on our switch
> and we have 5 nodes (10 ports in total).
>
> Thanks again for all you help
> Hossein
>
> > -- Hal
> >
> > > Many thanks for your kind help
> > > Hossein
> > >
> > > On Wed, Jun 06, 2007 at 09:09:53AM +0300, Tziporet Koren wrote:
> > > > Hossein Pourreza wrote:
> > > > >Hi all,
> > > > >
> > > > >I am new to infiniband stuff and am trying to configure an
> > > > > infiniband-based cluster using Linux FC 5. I downloaded the
> > > > > OFED-1.0 and tried to install it on
> > > > >cluster nodes. Now I can load the kernel modules without any error
> > > > > but I cannot
> > > > >run a simple test like ibv_ud_pingpong to check the connectivity of
> > > > > nodes in
> > > > >user-level.
> > > >
> > > > Have you run opensm?
> > > > You can run ibstat on each node to see ports are active
> > > >
> > > > Tziporet

-- 
Makia Minich <minich at ornl.gov>
National Center for Computation Science
Oak Ridge National Laboratory
--*--
Imagine no possessions
I wonder if you can
- John Lennon