[openib-general] EEH: MMIO Failure on Power5

[Thaddeus Ternes]

I'm attempting to bring up a Mellanox card in a Power5 machine and
have hit a snag.  I'm wondering if anybody else has seen issues
similar to this on this particular hardware, as these cards seem to
work in the Power4 machines.  The card is detected, but then I hit an
MMIO failure and ib_mthca fails.  The call trace (from dmesg) is
listed below.  I do see that the firmware is older, but am not sure if
that would necessarily bring about this problem.  Any input is
appreciated.

dmesg output:
[  138.477334] Freeing unused kernel memory: 360k freed
[  147.080672] ib_mthca: Mellanox InfiniBand HCA driver v0.06 (June 23, 2005)
[  147.080693] ib_mthca: Initializing Mellanox Technologies MT23108
InfiniHost (0001:c1:00.0)
[  147.081572] PCI: Enabling device: (0001:c1:00.0), cmd 142
[  148.355678] RTAS: event: 2, Type: Platform Error, Severity: 2
[  148.355689] EEH: MMIO failure (2) on device: pci15b3,5a44
/pci at 800000020000003/pci at 2/pci at 1/pci15b3,5a44 at 0
[  148.355709] Call Trace:
[  148.355717] [c0000003db0db050] [c00000000002fc80]
.eeh_dn_check_failure+0x2bc/0x314 (unreliable)
[  148.355745] [c0000003db0db130] [c00000000002fdd4]
.eeh_check_failure+0xfc/0x190
[  148.355765] [c0000003db0db1c0] [d0000000006557cc]
.mthca_cmd_poll+0x120/0x258 [ib_mthca]
[  148.355804] [c0000003db0db290] [d000000000655cc8]
.mthca_cmd_box+0x90/0xa8 [ib_mthca]
[  148.355839] [c0000003db0db330] [d000000000657444]
.mthca_INIT_HCA+0x240/0x288 [ib_mthca]
[  148.355877] [c0000003db0db3e0] [d000000000654790]
.mthca_init_one+0xd2c/0x180c [ib_mthca]
[  148.355913] [c0000003db0db870] [c0000000001d4a2c] .pci_device_probe+0xac/0xdc
[  148.355934] [c0000003db0db900] [c000000000239ec0]
.driver_probe_device+0x80/0x15c
[  148.355957] [c0000003db0db990] [c00000000023a130] .__driver_attach+0xa8/0xc4
[  148.355977] [c0000003db0dba20] [c0000000002390d4] .bus_for_each_dev+0x78/0xcc
[  148.355996] [c0000003db0dbad0] [c00000000023a174] .driver_attach+0x28/0x40
[  148.356016] [c0000003db0dbb50] [c000000000239848] .bus_add_driver+0xc8/0x1dc
[  148.356036] [c0000003db0dbc00] [c00000000023a7b0] .driver_register+0x44/0x5c
[  148.356056] [c0000003db0dbc90] [c0000000001d46e4]
.pci_register_driver+0x84/0xd8
[  148.356076] [c0000003db0dbd10] [d000000000669524]
.mthca_init+0x1c/0x48 [ib_mthca]
[  148.356122] [c0000003db0dbd90] [c00000000006cc88]
.sys_init_module+0x2f0/0x4cc
[  148.356143] [c0000003db0dbe30] [c00000000000d300] syscall_exit+0x0/0x18
[  148.356166] EEH: MMIO failure (2), notifiying device 0001:c1:00.0
Mellanox Technologies MT23108 InfiniHost
[  148.356247] ib_mthca 0001:c1:00.0: HCA FW version 3.2.0 is old
(3.3.3 is current).
[  148.356261] ib_mthca 0001:c1:00.0: If you have problems, try
updating your HCA FW.
[  148.357369] ib_mthca 0001:c1:00.0: SW2HW_MPT returned status 0x01
[  148.357382] ib_mthca 0001:c1:00.0: Failed to create driver PD, aborting.
[  148.359535] ib_mthca: probe of 0001:c1:00.0 failed with error -22

lsmod:
Module                  Size  Used by
ib_ipoib               56264  0
ib_sa                  19440  1 ib_ipoib
ib_mthca              150408  0
ib_mad                 53780  2 ib_sa,ib_mthca
ib_core                60704  4 ib_ipoib,ib_sa,ib_mthca,ib_mad

Thaddeus