***SPAM*** Re: [ofa-general] link width problem of Qlogic 9024 unmanaged switch
Yicheng Jia
YJia at tmriusa.com
Fri Apr 17 10:06:43 PDT 2009
Hi Ira,
Here is the output of "iblinkinfo.pl -R":
++++++++++++++++++++++++++++++++++++++++++++++++
[root at ib_manager ~]# iblinkinfo.pl -R
Switch 0x00066a00d90009c1 InfiniCon System InfinIO 9024 Lite:
7 1[ ] ==( 4X 2.5 Gbps Active / LinkUp)==> 6 1[ ]
"MT2520
4 InfiniHostLx Mellanox Technologies" ( )
2[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
3[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
4[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
5[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
6[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
7[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
8[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
9[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
10[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
11[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
12[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
13[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
14[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
15[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
16[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
17[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
18[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
19[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
20[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
21[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
22[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
23[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
24[ ] ==( 4X 2.5 Gbps Down / Polling)==> [ ]
"" ( )
++++++++++++++++++++++++++++++++++++++++++++++++++++
And the "ibstat" output:
+++++++++++++++++++++++++++++++++++
[root at ib_manager ~]# ibstat
CA 'mthca0'
CA type: MT25204
Number of ports: 1
Firmware version: 1.2.0
Hardware version: a0
Node GUID: 0x0002c90200230784
System image GUID: 0x0002c90200230787
Port 1:
State: Active
Physical state: LinkUp
Rate: 10
Base lid: 6
LMC: 0
SM lid: 6
Capability mask: 0x02500a6a
Port GUID: 0x0002c90200230785
++++++++++++++++++++++++++++++++++++++++++++++
The reset command I am using is "ibportstate 7 1 reset", I also tried
"ibportstate -D 0,1 1 reset", and it fails with the same result.
Thanks!
Yicheng Jia
Ira Weiny <weiny2 at llnl.gov>
04/16/2009 07:53 PM
To
Hal Rosenstock <hal.rosenstock at gmail.com>
cc
Yicheng Jia <YJia at tmriusa.com>, general at lists.openfabrics.org
Subject
Re: ***SPAM*** Re: [ofa-general] link width problem of Qlogic 9024
unmanaged switch
Yicheng,
I am hoping your system is small when I ask; could you send the output
from: "iblinkinfo.pl -R" When everything is up and running.
Also ibstat from the node you are attempting the reset on. As well as the
reset command you are using?
Thanks,
Ira
On Thu, 16 Apr 2009 19:12:00 -0400
Hal Rosenstock <hal.rosenstock at gmail.com> wrote:
> On Thu, Apr 16, 2009 at 6:06 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
> >
> >> There's a race condition here that I was asking about. If the link
> >> initialization takes too long and doesn't complete (gets to init)
> >> prior to the enable trying to be sent to the switch, then you could
> >> see these results but since it's DOWN until reboot it's something
> >> different.
> >
> > I did the "reset" when ports on both side of the link are in INIT
state and
> > LinkUp phys state.
> >
> >> If the disable/wait/enable worked that would've been another story.
> >
> > It fails too. Both ports go to DOWN after disable is issued and never
come
> > back. How long am I supposed to wait?
>
> Ideally you would see init before doing the enable but sounds like
> that's not occuring. Either you need low level debug to see why the
> link does not initialize at that point or get support from your
> CA/switch vendor(s). What's your CA device ?
>
> -- Hal
>
> > Thanks!
> > Yicheng Jia
> >
> >
> >
> >
> > Hal Rosenstock <hal.rosenstock at gmail.com>
> >
> > 04/16/2009 03:26 PM
> >
> > To
> > Yicheng Jia <YJia at tmriusa.com>
> > cc
> > Nicolas Morey-Chaisemartin <devel-ofed at morey-chaisemartin.com>,
> > general at lists.openfabrics.org
> > Subject
> > Re: [ofa-general] link width problem of Qlogic 9024 unmanaged switch
> >
> >
> >
> >
> > On Thu, Apr 16, 2009 at 3:47 PM, Yicheng Jia <YJia at tmriusa.com> wrote:
> >>
> >>> Are you resetting the switch from the peer HCA port or some other
port
> >>> ? That's what Nicolas asked but I might have missed the answer.
> >>
> >> Yes, I am trying to reset from the peer HCA port. Is anything wrong
with
> >> this?
> >
> > There's a race condition here that I was asking about. If the link
> > initialization takes too long and doesn't complete (gets to init)
> > prior to the enable trying to be sent to the switch, then you could
> > see these results but since it's DOWN until reboot it's something
> > different.
> >
> >>> Also, try disable (wait) and then enable and see if that works.
> >>
> >> It remains the same, the switch port is DOWN forever. No SMP massage
could
> >> get to the switch port.
> >
> > Right; in down, the SMP can't be sent.
> >
> >>> If I recall correctly, you had those links which are taking a long
time
> >>> to
> >>> initialize. If the link stays down forever after disable, this won't
> >>> work but I want to be sure.
> >>
> >> This is seperate issue.
> >
> > Since the link stays down yes. If the disable/wait/enable worked that
> > would've been another story.
> >
> > -- Hal
> >
> >> The "reset" command is tested on a single port HCA
> >> directly connected with Qlogic siwth. The HCA is plugged into a Linux
> >> machine. It is the simplest test environment.
> >>
> >> Thanks!
> >> Yicheng Jia
> >>
> >>
> >>
> >>
> >> Hal Rosenstock <hal.rosenstock at gmail.com>
> >>
> >> 04/16/2009 02:29 PM
> >>
> >> To
> >> Yicheng Jia <YJia at tmriusa.com>
> >> cc
> >> Nicolas Morey-Chaisemartin <devel-ofed at morey-chaisemartin.com>,
> >> general at lists.openfabrics.org
> >> Subject
> >> Re: [ofa-general] link width problem of Qlogic 9024 unmanaged switch
> >>
> >>
> >>
> >>
> >> On Thu, Apr 16, 2009 at 3:20 PM, Hal Rosenstock
> >> <hal.rosenstock at gmail.com> wrote:
> >>> On Thu, Apr 16, 2009 at 3:18 PM, Yicheng Jia <YJia at tmriusa.com>
wrote:
> >>>>
> >>>> They both are POLLING before "reset".
> >>>
> >>> Then they _should_ come back to INIT.
> >>>
> >>> What does the local LDDS value say after reset ? Any way to get the
> >>> switch port LDDS value ?
> >>
> >> Are you resetting the switch from the peer HCA port or some other
port
> >> ? That's what Nicolas asked but I might have missed the answer.
> >>
> >> Also, try disable (wait) and then enable and see if that works. If I
> >> recall correctly, you had those links which are taking a long time to
> >> initialize. If the link stays down forever after disable, this won't
> >> work but I want to be sure.
> >>
> >> -- Hal
> >>
> >>> -- Hal
> >>>
> >>>> Thanks!
> >>>> Yicheng Jia
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Hal Rosenstock <hal.rosenstock at gmail.com>
> >>>>
> >>>> 04/16/2009 01:53 PM
> >>>>
> >>>> To
> >>>> Yicheng Jia <YJia at tmriusa.com>
> >>>> cc
> >>>> Nicolas Morey-Chaisemartin <devel-ofed at morey-chaisemartin.com>,
> >>>> general at lists.openfabrics.org
> >>>> Subject
> >>>> Re: [ofa-general] link width problem of Qlogic 9024 unmanaged
switch
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Apr 16, 2009 at 11:12 AM, Yicheng Jia <YJia at tmriusa.com>
wrote:
> >>>>>
> >>>>> Hi Nicolas,
> >>>>>
> >>>>> After this "reset" command, both ports are DOWN forever, I can
only get
> >>>>> portinfo from local port.
> >>>>>
> >>>>> I am sure that the port that has been reset is not the local port,
> >>>>> otherwise
> >>>>> it will prompt "node type not switch" error.
> >>>>>
> >>>>> I tried to enable this switch port from another port and brought
it to
> >>>>> POLLING state, but as long as I use "reset", both ports are DOWN.
> >>>>
> >>>> What are the peer port's LinkDownDefaultStates ? Sounds like one or
> >>>> more must be Sleeping rather than Polling for some reason.
> >>>>
> >>>> -- Hal
> >>>>
> >>>>> Thanks!
> >>>>>
> >>>>> Yicheng Jia
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> Nicolas Morey-Chaisemartin <devel-ofed at morey-chaisemartin.com>
> >>>>>
> >>>>> 04/16/2009 12:43 AM
> >>>>>
> >>>>> To
> >>>>> Yicheng Jia <YJia at tmriusa.com>
> >>>>> cc
> >>>>> general at lists.openfabrics.org
> >>>>> Subject
> >>>>> Re: [ofa-general] link width problem of Qlogic 9024 unmanaged
switch
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> By any chances have you not reset the port you're on?
> >>>>> Have you tried using another node to enable the port again?
> >>>>>
> >>>>> Nicolas
> >>>>>
> >>>>> Le 16/04/2009 00:45, Yicheng Jia a écrit :
> >>>>>>
> >>>>>> Hello Randy,
> >>>>>>
> >>>>>> I am trying to run "ibportstate reset" to reset the switch port
on the
> >>>>>> other side in order to get 4x link. However I get the following
error:
> >>>>>> ibwarn: [19660] mad_rpc: _do_madrpc failed; dport (Lid 7)
> >>>>>> ibportstate: iberror: failed: smp set portinfo failed
> >>>>>>
> >>>>>> And the port status change to DOWN after this. Have you ever
tried to
> >>>>>> run "ibportstate" to reset the switch port?
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>> Yicheng Jia
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ------------------------------
> >>>>>>
> >>>>>> Message: 2
> >>>>>> Date: Wed, 4 Mar 2009 18:39:54 -0600
> >>>>>> From: Randy Halverson <randy.halverson at qlogic.com>
> >>>>>> Subject: [ofa-general] link width problem of Qlogic 9024
unmanaged
> >>>>>> switch
> >>>>>> To: "'general at lists.openfabrics.org'"
<general at lists.openfabrics.org>
> >>>>>> Message-ID:
> >>>>>> <88EC963376E93B4DB0F2A69D932F786903D89CAF at MNEXMB2.qlogic.org>
> >>>>>> Content-Type: text/plain; charset="us-ascii"
> >>>>>>
> >>>>>> Hello Yicheng,
> >>>>>>
> >>>>>> After checking internally, this appears to be a known problem
with
> >>>>>> older
> >>>>>> firmware for the 9024FC switches.
> >>>>>>
> >>>>>> It appears that you or another person at 'tmriusa.com' has
recently
> >>>>>> opened a case with QLogic Tech Support for this issue. Please
continue
> >>>>>> to work with QLogic Tech Support on firmware upgrade resolution
since
> >>>>>> you probably don't have our FastFabric Tools to manage the 9024FC
> >>>>>> switches..
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Randy
> >>>>>> Technical Support
> >>>>>> QLogic Corporation
> >>>>>> -------------- next part --------------
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
_____________________________________________________________________________
> >>>>>> Scanned by IBM Email Security Management Services powered by
> >>>>>> MessageLabs. For more information please visit http:// www.
ers.ibm.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
_____________________________________________________________________________
> >>>>>> <http:// www. ers.ibm.com/>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
_____________________________________________________________________________
> >>>>>> Scanned by IBM Email Security Management Services powered by
> >>>>>> MessageLabs. For more information please visit http:// www.
ers.ibm.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
_____________________________________________________________________________
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
------------------------------------------------------------------------
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> general mailing list
> >>>>>> general at lists.openfabrics.org
> >>>>>> http:// lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>>>>
> >>>>>> To unsubscribe, please visit
> >>>>>> http:// openib.org/mailman/listinfo/openib-general
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
_____________________________________________________________________________
> >>>>> Scanned by IBM Email Security Management Services powered by
> >>>>> MessageLabs.
> >>>>> For more information please visit http:// www. ers.ibm.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
_____________________________________________________________________________
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
_____________________________________________________________________________
> >>>>> Scanned by IBM Email Security Management Services powered by
> >>>>> MessageLabs.
> >>>>> For more information please visit http:// www. ers.ibm.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
_____________________________________________________________________________
> >>>>>
> >>>>> _______________________________________________
> >>>>> general mailing list
> >>>>> general at lists.openfabrics.org
> >>>>> http:// lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> >>>>>
> >>>>> To unsubscribe, please visit
> >>>>> http:// openib.org/mailman/listinfo/openib-general
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
_____________________________________________________________________________
> >>>> Scanned by IBM Email Security Management Services powered by
> >>>> MessageLabs.
> >>>> For more information please visit http:// www. ers.ibm.com
> >>>>
> >>>>
> >>>>
_____________________________________________________________________________
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
_____________________________________________________________________________
> >>>> Scanned by IBM Email Security Management Services powered by
> >>>> MessageLabs.
> >>>> For more information please visit http:// www. ers.ibm.com
> >>>>
> >>>>
> >>>>
_____________________________________________________________________________
> >>>>
> >>>
> >>
> >>
> >>
_____________________________________________________________________________
> >> Scanned by IBM Email Security Management Services powered by
MessageLabs.
> >> For more information please visit http:// www. ers.ibm.com
> >>
> >>
_____________________________________________________________________________
> >>
> >>
> >>
> >>
_____________________________________________________________________________
> >> Scanned by IBM Email Security Management Services powered by
MessageLabs.
> >> For more information please visit http:// www. ers.ibm.com
> >>
> >>
_____________________________________________________________________________
> >>
> >
> >
_____________________________________________________________________________
> > Scanned by IBM Email Security Management Services powered by
MessageLabs.
> > For more information please visit http:// www. ers.ibm.com
> >
_____________________________________________________________________________
> >
> >
> >
_____________________________________________________________________________
> > Scanned by IBM Email Security Management Services powered by
MessageLabs.
> > For more information please visit http:// www. ers.ibm.com
> >
_____________________________________________________________________________
> >
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http:// lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://
openib.org/mailman/listinfo/openib-general
>
--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
weiny2 at llnl.gov
_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs.
For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
_____________________________________________________________________________
Scanned by IBM Email Security Management Services powered by MessageLabs. For more information please visit http://www.ers.ibm.com
_____________________________________________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20090417/b5cd13c1/attachment.html>
More information about the general
mailing list