[ofa-general] Distributing MSI-X interrupts over multiple cores / CPUs ?

[Richard Frank]

Is it possible to have a single MSI-X vector serviced by multiple cores 
/ CPUs ?
 
If we set a single bit in the irq/smp_affinity mask we do see that 
interrupts
occur on the corresponding core / CPU.

However, if we set multiple bits - enabling multiple cores / CPUs - it 
appears that only the first
core (bit) in the map services interrupts..

Is this expected behavior ?

Are there any known issues / and or configuration steps that we must do 
for this to work ?

We are running a 2.6.18 kernel.. with OFED 1.3.1 and OFED 1.4.2 on 
multiple different Intel based
hardware platforms.

Tests with irqbalance disabled and enabled - have the same results.

# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       CPU4       
CPU5       CPU6       CPU7      
  0: 2427539089          0          0          0          0          
0          0          0    IO-APIC-edge  timer
  1:          2          0          0          0          0          
0          0          0    IO-APIC-edge  i8042
  6:          5          0          0          0          0          
0          0          0    IO-APIC-edge  floppy
  7:          2          0          0          0          0          
0          0          0    IO-APIC-edge  parport0
  8:          0          0          0          0          0          
0          0          0    IO-APIC-edge  rtc
  9:          0          0          0          0          0          
0          0          0   IO-APIC-level  acpi
 14:         60       9018          0          0   21238940        
180          0          0    IO-APIC-edge  ide0
 58:       7222       1282          0          0     588494        
458          0          0   IO-APIC-level  uhci_hcd:usb3, libata
 66:     663325      18918          0          0          0          
0          0          0       PCI-MSI-X  mlx4_core (async)
 74:    8887677   19753999          0          0          0          
0          0          0       PCI-MSI-X  mlx4_core (comp)
 82:        208          0          0          0          0          
0    5643776       4798         PCI-MSI  eth1
 90:         66          0          0          5    1232918          
0         68          0         PCI-MSI  eth2
169:         22          0          0          0          0          
0          0          0   IO-APIC-level  aic79xx
177:          0          0          0          0          0          
0          0          0   IO-APIC-level  uhci_hcd:usb4
185:         15          0          0          0          0          
0          0          0   IO-APIC-level  ehci_hcd:usb1, uhci_hcd:usb2, 
aic79xx
NMI:      20144       9082      11178      11356      10249       
8772      11643      10659
LOC: 2427207846 2427207756 2427207702 2427207613 2427207551 2427207458 
2427207407 2427207314