[ofa-general] Distributing MSI-X interrupts over multiple cores / CPUs ?

[Jason Gunthorpe]

On Tue, Sep 15, 2009 at 07:04:13PM -0400, Richard Frank wrote:
> We are running a 2.6.18 kernel.. with OFED 1.3.1 and OFED 1.4.2 on  
> multiple different Intel based
> hardware platforms.

Well, a quick glance at the MSI-X code in 2.6.18 shows the trouble, I
think:

        unsigned int dest_cpu = first_cpu(cpu_mask);

Probably breaks it good. dest_cpu ends up in the MSI register
programming and wrecks it. For most APIC configurations it should be a
bitmask of cpus that are participating.

Looks to my eyes like 2.6.31 is all fixed up. Alot of work has been
done in this area over the years. One major improvment since 2.6.18 is
that all the interrupt message programming for MSI, MSI-X and IO-APIC
is now centralized and far more uniform. So if it works for IO-APIC it
should work for MSI. Before it was a bit disjoint.

Jason