[ofw] Any interrupt coalescing in WinOF?

[Tzachi Dar]

Do you believe that interupt moderation will be able to help?

There is currently no way for the user can control that but I hope that this is a small change in the code.

By the way, are you running connect2 or connect?

Thanks
Tzachi

> -----Original Message-----
> From: Chris Worley [mailto:worleys at gmail.com]
> Sent: Tuesday, October 12, 2010 7:10 PM
> To: Tzachi Dar
> Cc: ofw at lists.openfabrics.org
> Subject: Re: [ofw] Any interrupt coalescing in WinOF?
> 
> On Tue, Oct 12, 2010 at 10:54 AM, Tzachi Dar <tzachid at mellanox.co.il>
> wrote:
> > There is no interrupt coalescing on winof although the hw does
> support that (I assume that you are running connectx).
> >
> > You can verify if the number of interrupts is big by running perfmon.
> Select the cpu, and choose number of interrupts per second.
> >
> > Look at this when your test is running and when it is not.
> 
> Although there are some Heisenberg issues w/ this test (perfmon locks
> up trying to measure cores that are pegged), I was able to confirm
> that all the interrupts are hitting these two cores, and assume each
> core is getting the interrupts for one HCA.
> 
> Thaks,
> 
> Chris
> >
> > Thanks
> > Tzachi
> >
> >> -----Original Message-----
> >> From: Chris Worley [mailto:worleys at gmail.com]
> >> Sent: Tuesday, October 12, 2010 6:49 PM
> >> To: Tzachi Dar
> >> Cc: ofw at lists.openfabrics.org
> >> Subject: Re: [ofw] Any interrupt coalescing in WinOF?
> >>
> >> On Tue, Oct 12, 2010 at 10:34 AM, Tzachi Dar
> <tzachid at mellanox.co.il>
> >> wrote:
> >> > What test/ulp are you using?
> >>
> >> IOMeter with one worker atop each of 4 SRP-imported/multipathed
> >> (round-robin) drives and a queue depth of 64.  I'm seeing ~450K
> random
> >> 4KB write IOPS.
> >>
> >> With two HCA's, both running the latest firmware (as of Friday), I
> see
> >> two cores on one NUMA node totally pegged/consumed, and affinity is
> >> set on the Dynamo's to avoid these cores... so something is either
> >> polling or interrupt handling on these.
> >>
> >> I'm running the latest 2.3RC4 WinOF.
> >>
> >> Thanks,
> >>
> >> Chris
> >> >
> >> > Thanks
> >> > Tzachi
> >> >
> >> >> -----Original Message-----
> >> >> From: ofw-bounces at lists.openfabrics.org [mailto:ofw-
> >> >> bounces at lists.openfabrics.org] On Behalf Of Chris Worley
> >> >> Sent: Tuesday, October 12, 2010 6:33 PM
> >> >> To: ofw at lists.openfabrics.org
> >> >> Subject: [ofw] Any interrupt coalescing in WinOF?
> >> >>
> >> >> I'm seeing the interrupt handlers completely consuming a CPU and
> >> >> becoming a bottleneck, and no interrupt load balancing across
> CPU's
> >> >> (except w/ multiple HCA's, each HCA consumes one CPU w/
> interrupts).
> >> >>
> >> >> Is interrupt coalescing manually tunable in WinOF?
> >> >>
> >> >> Or, am I totally off-base, and seeing polling and not interrupt
> >> >> behavior?
> >> >>
> >> >> Thanks,
> >> >>
> >> >> Chris
> >> >> _______________________________________________
> >> >> ofw mailing list
> >> >> ofw at lists.openfabrics.org
> >> >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
> >> >
> >