[openib-general] Start of an IPoIB FAQ

[Grant Grundler]

On Tue, Nov 23, 2004 at 01:20:21PM -0500, Hal Rosenstock wrote:
> Hi,
> 
> The start of an IPoIB FAQ may be in order.

Yes - good idea. I need it too.

> Something along the lines of:
> 
> ping doesn't work between IPoIB nodes. What should I do ?
> 
> First, verify that the ports are active. This can be done via:
> 
> cat /sys/class/infiniband/mthca0/ports/1/state

cat: /sys/class/infiniband/mthca0/ports/1/state: No such file or directory
gsyprf3:~# ls /sys/class/infiniband/  
gsyprf3:~# lsmod
Module                  Size  Used by
ib_ipoib              104344  0 
ib_sa                  24620  1 ib_ipoib
ipt_state               5528  13 
ib_mthca              168167  0 
ib_mad                 60352  2 ib_sa,ib_mthca
ib_core                81328  4 ib_ipoib,ib_sa,ib_mthca,ib_mad
gsyprf3:~# lspci -vs 81:0.0
0000:81:00.0 InfiniBand: Mellanox Technology MT23108 InfiniHost (rev a1)
        Subsystem: Mellanox Technology MT23108 InfiniHost
	Flags: 66MHz, medium devsel, IRQ 67
	Memory at 00000000cf700000 (64-bit, non-prefetchable) [size=1M]
	Memory at 00000000cf800000 (64-bit, prefetchable) [size=8M]
	Memory at 00000000d0000000 (64-bit, prefetchable) [size=256M]
	Capabilities: [40] #11 [001f]
	Capabilities: [50] Vital Product Data
	Capabilities: [60] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable-
	Capabilities: [70] PCI-X non-bridge device.

Maybe trying to use an unsupported card?
Ah yes...something along that line:
ib_mthca: Mellanox InfiniBand HCA driver v0.06-pre (November 8, 2004)
ib_mthca: Initializing Mellanox Technology MT23108 InfiniHost (0000:81:00.0)
GSI 60 (level, low) -> CPU 0 (0x0000) vector 67
ACPI: PCI interrupt 0000:81:00.0[A] -> GSI 60 (level, low) -> IRQ 67
ib_mthca 0000:81:00.0: Unhandled event 0f(00) on eqn 3
ib_query_gid failed (-16) for mthca0 (index 12)
ib_query_port failed (-16) for mthca0
ib_mthca 0000:81:00.0: WRITE_MTT failed (-16)
ib_mad: Couldn't create ib_mad CQ
ib_mad: Couldn't open mthca0 port 1

I can debug this a bit more later today.

In anycase, the FAQ sounds like a great idea.

grant