[ofa-general] IB performance stats (revisited)

[Hal Rosenstock]

On Wed, 2007-06-27 at 14:22, Jason Gunthorpe wrote:
> On Wed, Jun 27, 2007 at 01:48:04PM -0400, Hal Rosenstock wrote:
> 
> > Another approach would be to have the PMA inform the kernel that the
> > counters were reset (perhaps including the values prior to the reset) so
> > that these could be factored into the local set of counters. There is
> > nothing in the spec that precludes this although it has not been
> > implemented this way. Then there would't be a reason for a local manager
> > to have to play these games. It would mean that there would need to be a
> > performance manager running in the subnet which may not be acceptable
> > for some installations; not sure.
> 
> If you are going to play those sorts of games I think it would better
> to just effectively disable the PMA in the mellanox firmware and do
> the following:
> 
> - The kernel periodically fetches the performance stats and aggregates
>   them into a 64 wrapping counter. The kernel sends PMA mads into the
>   mellanox firmware to read and reset the counters
> - The new 64 bit stats are exported via sysfs/proc/whatever as
>   wrapping counters
> - When a PMA packet comes in the kernel services it rather than
>   passing it on to the chip firmware.

In this way, both 32 and 64 bit counters could be presented by the PMA
but how would it know when the a counter has maxed out in terms of the
PMA and how would a remote clear be handled ?

-- Hal

> Hopefully in future we could encourage new firmware/sillicon to
> support exporting non-wrapping 64 bit counters to the OS so this ugly
> mess wouldn't be needed.
> 
> FWIW, I agree with Mark that the current locally accessible counters
> that are exactly the same as PMA mad values are virtually useless..
> 
> Jason