[openib-general] Re: newbie to openib
Lamia M.Youseff
lyouseff at cs.ucsb.edu
Tue Feb 14 12:15:07 PST 2006
>What does lspci -vv show?
>
>Is the ib_mthca module loaded?
>
>BTW, when debugging problems, you should probably set
>CONFIG_INFINIBAND_MTHCA_DEBUG=y.
>
> - R.
>
>
Thanks Roland for your reply. I recompiled the kernel with infiniband
debugging. After making sure lib_mthca module is load, I tried ibstatus.
It seems that the tool is looking for some file in
/sys/class/infiniband/*/ports. However, this infiniband directory is
empty. The output of lspci -vv shows a PCI-EXPRESS card is there, which
i believe is the infiniband card. However, nothing about mthca or hca. I
have the output of those commands below.
That is getting even more confusing for me.
-- Lamia
-bash-3.00# lsmod
Module Size Used by
ib_ucm 18296 0
ib_cm 33904 1 ib_ucm
ib_uverbs 30872 0
ib_umad 15776 0
ib_mthca 98848 0
ib_mad 39208 3 ib_cm,ib_umad,ib_mthca
ib_core 44544 5 ib_cm,ib_uverbs,ib_umad,ib_mthca,ib_mad
-bash-3.00# ibstatus
Fatal error: device '*': sys files not found (/sys/class/infiniband/*/ports)
-bash-3.00# ibv_devinfo
libibverbs: Fatal: no infiniband class devices found.
No IB devices found
-bash-3.00# lspci
00:00.0 Host bridge: Intel Corporation E7525 Memory Controller Hub (rev 0a)
00:02.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express Port A (rev 0a)
00:03.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express Port A1 (rev 0a)
00:04.0 PCI bridge: Intel Corporation E7525/E7520 PCI Express Port B (rev 0a)
00:1d.0 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #3 (rev 02)
00:1d.3 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #4 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB2 EHCI Controller (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c2)
00:1f.0 ISA bridge: Intel Corporation 82801EB/ER (ICH5/ICH5R) LPC Interface Bridge (rev 02)
00:1f.2 IDE interface: Intel Corporation 82801EB (ICH5) SATA Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801EB/ER (ICH5/ICH5R) SMBus Controller (rev 02)
00:1f.5 Multimedia audio controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) AC'97 Audio Controller (rev 02)
02:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A (rev 09)
02:00.1 PIC: Intel Corporation 6700/6702PXH I/OxAPIC Interrupt Controller A (rev 09)
02:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B (rev 09)
02:00.3 PIC: Intel Corporation 6700PXH I/OxAPIC Interrupt Controller B (rev 09)
04:02.0 Ethernet controller: Intel Corporation 82545GM Gigabit Ethernet Controller (rev 04)
-bash-3.00# lspci -v
00:00.0 Host bridge: Intel Corporation E7525 Memory Controller Hub (rev 0a)
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, fast devsel, latency 0
Capabilities: [40] Vendor Specific Information
00:02.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express Port A (rev 0a) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
Capabilities: [50] Power Management version 2
Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable-
Capabilities: [64] Express Root Port (Slot-) IRQ 0
00:03.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express Port A1 (rev 0a) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=02, subordinate=04, sec-latency=0
I/O behind bridge: 00002000-00002fff
Memory behind bridge: dfb00000-dfcfffff
Capabilities: [50] Power Management version 2
Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable-
Capabilities: [64] Express Root Port (Slot-) IRQ 0
00:04.0 PCI bridge: Intel Corporation E7525/E7520 PCI Express Port B (rev 0a) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
Capabilities: [50] Power Management version 2
Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable-
Capabilities: [64] Express Root Port (Slot-) IRQ 0
00:1d.0 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #1 (rev 02) (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, medium devsel, latency 0, IRQ 145
I/O ports at 1840 [size=32]
00:1d.1 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #2 (rev 02) (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, medium devsel, latency 0, IRQ 169
I/O ports at 1860 [size=32]
00:1d.2 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #3 (rev 02) (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, medium devsel, latency 0, IRQ 161
I/O ports at 1880 [size=32]
00:1d.3 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB UHCI Controller #4 (rev 02) (prog-if 00 [UHCI])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, medium devsel, latency 0, IRQ 145
I/O ports at 18a0 [size=32]
00:1d.7 USB Controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) USB2 EHCI Controller (rev 02) (prog-if 20 [EHCI])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, medium devsel, latency 0, IRQ 177
Memory at dfa00000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Debug port
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c2) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=06, subordinate=06, sec-latency=32
00:1f.0 ISA bridge: Intel Corporation 82801EB/ER (ICH5/ICH5R) LPC Interface Bridge (rev 02)
Flags: bus master, medium devsel, latency 0
00:1f.2 IDE interface: Intel Corporation 82801EB (ICH5) SATA Controller (rev 02) (prog-if 8a [Master SecP PriP])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, 66Mhz, medium devsel, latency 0, IRQ 161
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at 18f0 [size=16]
00:1f.3 SMBus: Intel Corporation 82801EB/ER (ICH5/ICH5R) SMBus Controller (rev 02)
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: medium devsel, IRQ 153
I/O ports at 1100 [size=32]
00:1f.5 Multimedia audio controller: Intel Corporation 82801EB/ER (ICH5/ICH5R) AC'97 Audio Controller (rev 02)
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, medium devsel, latency 0, IRQ 153
I/O ports at 1400 [size=256]
I/O ports at 1800 [size=64]
Memory at dfa00c00 (32-bit, non-prefetchable) [size=512]
Memory at dfa00800 (32-bit, non-prefetchable) [size=256]
Capabilities: [50] Power Management version 2
02:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A (rev 09) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=03, subordinate=03, sec-latency=64
Capabilities: [44] Express PCI/PCI-X Bridge IRQ 0
Capabilities: [5c] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Capabilities: [6c] Power Management version 2
Capabilities: [d8] PCI-X bridge device.
02:00.1 PIC: Intel Corporation 6700/6702PXH I/OxAPIC Interrupt Controller A (rev 09) (prog-if 20 [IO(X)-APIC])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, fast devsel, latency 0
Memory at dfb00000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Express Endpoint IRQ 0
Capabilities: [6c] Power Management version 2
02:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B (rev 09) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=02, secondary=04, subordinate=04, sec-latency=64
I/O behind bridge: 00002000-00002fff
Memory behind bridge: dfc00000-dfcfffff
Capabilities: [44] Express PCI/PCI-X Bridge IRQ 0
Capabilities: [5c] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Capabilities: [6c] Power Management version 2
Capabilities: [d8] PCI-X bridge device.
02:00.3 PIC: Intel Corporation 6700PXH I/OxAPIC Interrupt Controller B (rev 09) (prog-if 20 [IO(X)-APIC])
Subsystem: Super Micro Computer Inc: Unknown device 5680
Flags: bus master, fast devsel, latency 0
Memory at dfb01000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [44] Express Endpoint IRQ 0
Capabilities: [6c] Power Management version 2
04:02.0 Ethernet controller: Intel Corporation 82545GM Gigabit Ethernet Controller (rev 04)
Subsystem: Intel Corporation PRO/1000 MT Server Adapter
Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 185
Memory at dfc00000 (64-bit, non-prefetchable) [size=128K]
I/O ports at 2000 [size=64]
Capabilities: [dc] Power Management version 2
Capabilities: [e4] PCI-X non-bridge device.
Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
-bash-3.00# lspci -vv
00:02.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express Port A (rev
0a) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Step
ping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR+ <PERR-
Latency: 0, Cache Line Size 08
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
Secondary status: 66Mhz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot
+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable
-
Address: fee00000 Data: 0000
Capabilities: [64] Express Root Port (Slot-) IRQ 0
Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag-
Device: Latency L0s <64ns, L1 <1us
Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 128 bytes, MaxReadReq 128 bytes
Link: Supported Speed 2.5Gb/s, Width x0, ASPM L0s, Port 2
Link: Latency L0s <4us, L1 unlimited
Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x0
Root: Correctable- Non-Fatal- Fatal- PME-
00:03.0 PCI bridge: Intel Corporation E7525/E7520/E7320 PCI Express Port A1 (rev
0a) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Step
ping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR-
Latency: 0, Cache Line Size 08
Bus: primary=00, secondary=02, subordinate=04, sec-latency=0
I/O behind bridge: 00002000-00002fff
Memory behind bridge: dfb00000-dfcfffff
Secondary status: 66Mhz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot
+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable
-
Address: fee00000 Data: 0000
Capabilities: [64] Express Root Port (Slot-) IRQ 0
Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag-
Device: Latency L0s <64ns, L1 <1us
Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 256 bytes, MaxReadReq 512 bytes
Link: Supported Speed 2.5Gb/s, Width x4, ASPM L0s, Port 3
Link: Latency L0s <4us, L1 unlimited
Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x4
Root: Correctable- Non-Fatal- Fatal- PME-
00:04.0 PCI bridge: Intel Corporation E7525/E7520 PCI Express Port B (rev 0a) (p
rog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Step
ping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR+ <PERR-
Latency: 0, Cache Line Size 08
Bus: primary=00, secondary=05, subordinate=05, sec-latency=0
Secondary status: 66Mhz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot
+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] Message Signalled Interrupts: 64bit- Queue=0/1 Enable
-
Address: fee00000 Data: 0000
Capabilities: [64] Express Root Port (Slot-) IRQ 0
Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag-
Device: Latency L0s <64ns, L1 <1us
Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 128 bytes, MaxReadReq 128 bytes
Link: Supported Speed 2.5Gb/s, Width x0, ASPM L0s, Port 4
Link: Latency L0s <4us, L1 unlimited
Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x0
Root: Correctable- Non-Fatal- Fatal- PME-
...............
02:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A (rev 0
9) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Step
ping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR-
Latency: 0, Cache Line Size 10
Bus: primary=02, secondary=03, subordinate=03, sec-latency=64
Secondary status: 66Mhz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort
- <MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
Capabilities: [44] Express PCI/PCI-X Bridge IRQ 0
Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag-
Device: Latency L0s <64ns, L1 <1us
Device: AtnBtn- AtnInd- PwrInd-
Device: Errors: Correctable- Non-Fatal- Fatal+ Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 256 bytes, MaxReadReq 512 bytes
Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 0
Link: Latency L0s unlimited, L1 unlimited
Link: ASPM Disabled CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x8
Capabilities: [5c] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable
-
Address: 0000000000000000 Data: 0000
Capabilities: [6c] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot
+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [d8] PCI-X bridge device.
Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-, SRD- Freq=2
Status: Bus=2 Dev=0 Func=0 64bit- 133MHz- SCD- USC-, SCO-, SRD-
: Upstream: Capacity=65535, Commitment Limit=65535
: Downstream: Capacity=65535, Commitment Limit=65535
...............
02:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B (rev 09) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Latency: 0, Cache Line Size 10
Bus: primary=02, secondary=04, subordinate=04, sec-latency=64
I/O behind bridge: 00002000-00002fff
Memory behind bridge: dfc00000-dfcfffff
Secondary status: 66Mhz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
Capabilities: [44] Express PCI/PCI-X Bridge IRQ 0
Device: Supported: MaxPayload 256 bytes, PhantFunc 0, ExtTag-
Device: Latency L0s <64ns, L1 <1us
Device: AtnBtn- AtnInd- PwrInd-
Device: Errors: Correctable- Non-Fatal- Fatal+ Unsupported-
Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
Device: MaxPayload 256 bytes, MaxReadReq 512 bytes
Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 0
Link: Latency L0s unlimited, L1 unlimited
Link: ASPM Disabled CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x8
Capabilities: [5c] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
Address: 0000000000000000 Data: 0000
Capabilities: [6c] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [d8] PCI-X bridge device.
Secondary Status: 64bit+, 133MHz+, SCD-, USC-, SCO-, SRD- Freq=3
Status: Bus=2 Dev=0 Func=2 64bit- 133MHz- SCD- USC-, SCO-, SRD-
: Upstream: Capacity=65535, Commitment Limit=65535
: Downstream: Capacity=65535, Commitment Limit=65535
.................
-bash-3.00# lspci -vv | grep hca
-bash-3.00# lspci -vv | grep HCA
-bash-3.00# lspci -vv | grep MTHCA
-bash-3.00# lspci -vv | grep mthca
More information about the general
mailing list