[ofa-general] SRP/mlx4 interrupts throttling performance

[Cameron Harr]

Vladislav Bolkhovitin wrote:
> Cameron Harr wrote:
>> Vlad,
>> Thanks for the suggestion. As I look via vmstat, my CSw/s rate is 
>> fairly constant around 280K when scst_threads=1 (per Vu's suggestion) 
>> and pops up to ~330-340K CSw/s when scst_threads is set to 8. I'm 
>> currently doing 512B writes, and this gives me about a 4:1 ratio of 
>> context switches to IOPs with 1 SCST thread (70K IOPs) and around 
>> 4.5:1 when there are 8 SCST threads (75K IOPs).
>
> This is still too high. Considering that each CS is about 1 
> microsecond you can estimate how many IOPS's it costs you.

Dropping scst_threads down to 2, from 8, with 2 initiators, seems to 
make a fairly significant difference, propelling me to a little over 
100K IOPs and putting the CS rate around 2:1, sometimes lower. 2 threads 
gave the best performance compared to 1, 4 and 8.