[Users] ofed and ubuntu server

[Weiny, Ira]

Excellent news!

Ira

From: Andrei Mikhailovsky [mailto:andrei at arhont.com]
Sent: Tuesday, July 15, 2014 8:06 AM
To: Weiny, Ira
Cc: Rupert Dance; Jack Hill; users at lists.openfabrics.org
Subject: Re: [Users] ofed and ubuntu server

Ira,

I think i've made some progress.

In the past, I had the following when trying to run ibstat:


ibstat

CA 'qib0'

        CA type: InfiniPath_QMH7342

        Number of ports: 2

        Firmware version:

        Hardware version: 2

        Node GUID: 0x00117500007965b4

        System image GUID: 0x00117500007965b4

        Port 1:

                State: Down

                Physical state: Disabled

                Rate: 40

                Base lid: 65535

                LMC: 0

                SM lid: 65535

                Capability mask: 0x07600868

                Port GUID: 0x00117500007965b4

                Link layer: InfiniBand

        Port 2:

                State: Down

                Physical state: Disabled

                Rate: 40

                Base lid: 65535

                LMC: 0

                SM lid: 65535

                Capability mask: 0x07600868

                Port GUID: 0x00117500007965b5

                Link layer: InfiniBand



as you can see, the physical state is shown as Disabled.



This was the case for both Ubuntu 12.04 (using roland dreier pap) and 14.04 using stock infiniband tools.



I couldn't figure out what's going on as the exact estup with Mellanox card works perfectly well.



I was just looking at the startup scripts for the openibd script that comes with qlogic/intel drivers and i've noticed that the script starts a file called /etc/infiniband/truescale.cmds. I've copied the script across from a centos server and started it. And now I have a link shown as ACTIVE and I can see other nodes when running iblinkinfo.



So, I will now try to get ipoib working.



Cheers





--
Andrei Mikhailovsky
Director
Arhont Information Security

Web: http://www.arhont.com
http://www.wi-foo.com
Tel: +44 (0)870 4431337
Fax: +44 (0)208 429 3111
PGP: Key ID - 0x2B3438DE
PGP: Server - keyserver.pgp.com

DISCLAIMER

The information contained in this email is intended only for the use of the person(s) to whom it is addressed and may be confidential or contain legally privileged information. If you are not the intended recipient you are hereby notified that any perusal, use, distribution, copying or disclosure is strictly prohibited. If you have received this email in error please immediately advise us by return email at andrei at arhont.com<mailto:andrei at arhont.com> and delete and purge the email and any attachments without making a copy.


________________________________
From: "Ira Weiny" <ira.weiny at intel.com<mailto:ira.weiny at intel.com>>
To: "Rupert Dance" <rsdance at soft-forge.com<mailto:rsdance at soft-forge.com>>, "Andrei Mikhailovsky" <andrei at arhont.com<mailto:andrei at arhont.com>>, "Jack Hill" <jackhill at unc.edu<mailto:jackhill at unc.edu>>
Cc: "users at lists.openfabrics.org<mailto:users at lists.openfabrics.org>" <users at lists.openfabrics.org<mailto:users at lists.openfabrics.org>>
Sent: Tuesday, 15 July, 2014 3:37:46 PM
Subject: RE: [Users] ofed and ubuntu server

Infinipath-PSM is not needed to get the link to go active. If the link is not active what is it? (Ibstat output)

Could you provide a bit more info on your configuration?

Versions of the following
Kernel on each node
Opensm
Infinibad-diags

Node information
Sm node config (mlx or Qib)
Any switches between the sm node and the Qib node
Qib node config
"Sminfo" output
What does the sm log say about that node or the switch port it is connected to?

Finally can you talk to the failing node with smpquery from the sm node?

Ira


-----Original Message-----
From: Rupert Dance [rsdance at soft-forge.com<mailto:rsdance at soft-forge.com>]
Sent: Tuesday, July 15, 2014 06:33 AM Pacific Standard Time
To: 'Andrei Mikhailovsky'; 'Jack Hill'
Cc: users at lists.openfabrics.org<mailto:users at lists.openfabrics.org>
Subject: Re: [Users] ofed and ubuntu server
Andrei,

There is a known issue with infinipath-psm and later generations of the gcc compiler. Perhaps it is related to your issue. Please check out this OFED Bug 2475:

http://bugs.openfabrics.org/bugzilla/show_bug.cgi?id=2475

Thanks

Rupert



From: users-bounces at lists.openfabrics.org<mailto:users-bounces at lists.openfabrics.org> [mailto:users-bounces at lists.openfabrics.org] On Behalf Of Andrei Mikhailovsky
Sent: Tuesday, July 15, 2014 7:14 AM
To: Jack Hill
Cc: users at lists.openfabrics.org<mailto:users at lists.openfabrics.org>
Subject: Re: [Users] ofed and ubuntu server

Jack,

Thanks for your reply. I was having issues getting the Active link status on those adapters. I've tried a bunch of things, but nothing helped me. I think the issues was with infinipath-psm as I couldnt get it compiled. Did you have any problems with compiling infinipath-psm? In a contrast, I've had absolutely no issues with the Mellanox gear, which worked like a charm )))

Andrei

________________________________
From: "Jack Hill" <jackhill at unc.edu<mailto:jackhill at unc.edu>>
To: "Andrei Mikhailovsky" <andrei at arhont.com<mailto:andrei at arhont.com>>
Cc: users at lists.openfabrics.org<mailto:users at lists.openfabrics.org>
Sent: Monday, 14 July, 2014 3:00:13 PM
Subject: Re: [Users] ofed and ubuntu server

On Mon, 14 Jul 2014, Andrei Mikhailovsky wrote:

> I was wondering if there is any progress in supporting Ubuntu servers 12.04 or 14.04? I am having
> a few issues getting my Qlogic 7322 QDR to work on Ubuntu Servers. It works fine on red hat based
> distros, but not on Ubuntu ((

Andrei,

What issues are you having? I've been using those adapters with Ubuntu
12.04 with distribution supplied packages. I did install infinipath-psm
from source to support some MPI operations.

Best,
Jack


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20140715/a17c4ce5/attachment.html>