[ofa-general] Re: [PATCH 2 of 3] IB/mthca: support multiple completion vectors

[Rick Jones]

Michael S. Tsirkin wrote:
>>Quoting Roland Dreier <rdreier at cisco.com>:
>>Subject: Re: [PATCH 2 of 3] IB/mthca: support multiple completion vectors
>>
>> > I don't know how many vectors make sense, so I decided to be
>> > conservative here, since each EQ consumes a lot of memory by default.
>>
>>I think #vectors == O(#CPUs) is what we should aim for.
>>
>>Also another useful thing for NUMA systems might be to allocate the EQ
>>memory from the CPU where that interrupt will be targeted.
> 
> 
> Can't interrupts migrate between CPUs?

Only if someone leaves the irqbalancer running.  Given that it isn't presently 
NUMA-aware (plusungood) I at least tend to disable it.  Apart from it, then 
generally only under explicit administrator command would an interrupt be 
migrated from one CPU to another.

> 
> OTOH, especially for latency, it might be best to only have 1 interrupt,
> and service that on the node closest to device.
> 

topological proximity is a good thing.  there can be more than 1 core 
"topologically close" to the I/O card.

rickjones