[openib-general] Help with CONFIG_PCI_MSI in the kernel
Jason Gunthorpe
jgunthorpe at obsidianresearch.com
Tue Apr 4 11:07:23 PDT 2006
On Tue, Apr 04, 2006 at 09:45:09AM -0700, Grant Grundler wrote:
> > MSI requires end device support and something in a bridge
> > to transform the MSI message into an APIC message, but the kernel
> > currently only looks for end device support.
>
> APIC Message?
> MSI is just a DMA-write from the card point of view.
> So if PCI is working and DMA is working, MSI should work too.
> The difference is routing of the transaction and the fact that
> it's not targeting Host memory but some other part of the chipset.
You still need chipset support to get from the memory write to an
interrupt message transaction on the FSB to the processor APIC.
For an example of how intel documents this in their chipsets see
http://www.intel.com/design/chipsets/datashts/30146403.pdf
Pages 19 and 165
I doubt all intel chipsets ever produced have this transformation. I
also know that alot of embedded systems don't have host bridges that
support this. The list of host bridges that work is definately smaller
than the list that don't. :<
> > Presumably only some subset of Intel PCI/PCI-X bridges support MSI
> > translation to an APIC message on the FSB, and we know on HT systems
> > that the MSI PCI capability block is required (and must be *enabled*)
> > in the PCI/PCI-X/PCIe bridge for there to be support.
> Well, can't linux enable that block if it's present?
> That isn't a reason to disable MSI for _all_ systems.
I have a patch that does that, but monkying with the memory map always
makes me nervous since you never really know what the BIOS has
done. Intel style MSI's overlap the high memory BIOS area so there is
a potential problem. HT MSI translation can be configured to use a
high address, so it might be very safe to enable the translation and
set something >4G as the base.
> > In the long term as PCIe becomes dominant this is best since linux
> > becomes firmly in control of all interrupt routing and doesn't rely on
> > working APCI for IOAPIC tables..
> Sorry...I can't agree.
> Line based interrupt routing is dependent on firmware to give
> us the IRQ->APIC routing tables, enough info to identify CPUs (ID/EID
> info for Intel implementations) and program IO-xAPIC entries.
> Essentially, MSI only needs the CPU Info so MSI transactions get
> routed correctly. Then MSI/-X entries on the devices can be
> programmed (essentially the same way an IO-xAPIC gets programmed).
?? That's what I was trying to say - on a system with only PCIe
(granted, with working MSI in the devices..) there should be limited
need for IOAPIC based routing.
> If it's "just a BIOS" issue, then can't AMD help linux turn MSI support on?
> ie linux can bang some values into the chipset like we do for other types
> of initialization when BIOS doesn't do it right.
Yes, simple patch attached..
Jason
-------------- next part --------------
--- linux-2.6.15.4/drivers/pci/quirks.c 2006-02-16 12:08:59.000000000 -0700
+++ lin/drivers/pci/quirks.c 2006-02-16 12:12:30.000000000 -0700
@@ -1257,6 +1257,29 @@
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_NCR, PCI_DEVICE_ID_NCR_53C810, fixup_rev1_53c810);
+#ifdef CONFIG_PCI_MSI
+static void __devinit fixup_ht_msi(struct pci_dev* dev)
+{
+ /* Some BIOS's do not enable the hypertransport MSI mapping capability
+ on the chipset. This breaks MSI support.. */
+ int pos = pci_find_capability(dev,PCI_CAP_ID_HT);
+ while (pos != 0)
+ {
+ u32 cap;
+ pci_read_config_dword(dev,pos,&cap);
+ if (((cap >> 16) & PCI_HT_CMD_TYP) == PCI_HT_CMD_TYP_MSIM) {
+ if ((cap & PCI_HT_MSIM_ENABLE) == 0) {
+ printk("BIOS BUG: HyperTransport MSI mapping not enabled for %s, enabling.\n",pci_name(dev));
+ cap |= PCI_HT_MSIM_ENABLE;
+ pci_write_config_dword(dev,pos,cap);
+ }
+ break;
+ }
+ pos = pci_find_next_capability(dev, pos, PCI_CAP_ID_HT);
+ }
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID, PCI_ANY_ID, fixup_ht_msi);
+#endif
static void pci_do_fixups(struct pci_dev *dev, struct pci_fixup *f, struct pci_fixup *end)
{
--- linux-2.6.15.4/include/linux/pci_regs.h 2006-02-16 12:09:05.000000000 -0700
+++ lin/include/linux/pci_regs.h 2006-02-16 12:12:30.000000000 -0700
@@ -196,12 +196,14 @@
#define PCI_CAP_ID_MSI 0x05 /* Message Signalled Interrupts */
#define PCI_CAP_ID_CHSWP 0x06 /* CompactPCI HotSwap */
#define PCI_CAP_ID_PCIX 0x07 /* PCI-X */
+#define PCI_CAP_ID_HT 0x08 /* HyperTransport */
#define PCI_CAP_ID_SHPC 0x0C /* PCI Standard Hot-Plug Controller */
#define PCI_CAP_ID_EXP 0x10 /* PCI Express */
#define PCI_CAP_ID_MSIX 0x11 /* MSI-X */
#define PCI_CAP_LIST_NEXT 1 /* Next capability in the list */
#define PCI_CAP_FLAGS 2 /* Capability defined flags (16 bits) */
#define PCI_CAP_SIZEOF 4
+#define PCI_HT_CMD_TYP 0xf800 /* Hypertransport capability type mask */
/* Power Management Registers */
@@ -285,6 +287,10 @@
#define PCI_MSI_DATA_64 12 /* 16 bits of data for 64-bit devices */
#define PCI_MSI_MASK_BIT 16 /* Mask bits register */
+/* HyperTransport MSI Mapping registers */
+#define PCI_HT_CMD_TYP_MSIM 0xa800 // MSI Mapping type
+#define PCI_HT_MSIM_ENABLE (1<<16)
+
/* CompactPCI Hotswap Register */
#define PCI_CHSWP_CSR 2 /* Control and Status Register */
More information about the general
mailing list