[ofw] Any interrupt coalescing in WinOF?

[Chris Worley]

On Tue, Oct 12, 2010 at 10:54 AM, Tzachi Dar <tzachid at mellanox.co.il> wrote:
> There is no interrupt coalescing on winof although the hw does support that (I assume that you are running connectx).
>
> You can verify if the number of interrupts is big by running perfmon. Select the cpu, and choose number of interrupts per second.
>
> Look at this when your test is running and when it is not.

Although there are some Heisenberg issues w/ this test (perfmon locks
up trying to measure cores that are pegged), I was able to confirm
that all the interrupts are hitting these two cores, and assume each
core is getting the interrupts for one HCA.

Thaks,

Chris
>
> Thanks
> Tzachi
>
>> -----Original Message-----
>> From: Chris Worley [mailto:worleys at gmail.com]
>> Sent: Tuesday, October 12, 2010 6:49 PM
>> To: Tzachi Dar
>> Cc: ofw at lists.openfabrics.org
>> Subject: Re: [ofw] Any interrupt coalescing in WinOF?
>>
>> On Tue, Oct 12, 2010 at 10:34 AM, Tzachi Dar <tzachid at mellanox.co.il>
>> wrote:
>> > What test/ulp are you using?
>>
>> IOMeter with one worker atop each of 4 SRP-imported/multipathed
>> (round-robin) drives and a queue depth of 64.  I'm seeing ~450K random
>> 4KB write IOPS.
>>
>> With two HCA's, both running the latest firmware (as of Friday), I see
>> two cores on one NUMA node totally pegged/consumed, and affinity is
>> set on the Dynamo's to avoid these cores... so something is either
>> polling or interrupt handling on these.
>>
>> I'm running the latest 2.3RC4 WinOF.
>>
>> Thanks,
>>
>> Chris
>> >
>> > Thanks
>> > Tzachi
>> >
>> >> -----Original Message-----
>> >> From: ofw-bounces at lists.openfabrics.org [mailto:ofw-
>> >> bounces at lists.openfabrics.org] On Behalf Of Chris Worley
>> >> Sent: Tuesday, October 12, 2010 6:33 PM
>> >> To: ofw at lists.openfabrics.org
>> >> Subject: [ofw] Any interrupt coalescing in WinOF?
>> >>
>> >> I'm seeing the interrupt handlers completely consuming a CPU and
>> >> becoming a bottleneck, and no interrupt load balancing across CPU's
>> >> (except w/ multiple HCA's, each HCA consumes one CPU w/ interrupts).
>> >>
>> >> Is interrupt coalescing manually tunable in WinOF?
>> >>
>> >> Or, am I totally off-base, and seeing polling and not interrupt
>> >> behavior?
>> >>
>> >> Thanks,
>> >>
>> >> Chris
>> >> _______________________________________________
>> >> ofw mailing list
>> >> ofw at lists.openfabrics.org
>> >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>> >
>