[ofa-general] IB performance stats (revisited)

[Jason Gunthorpe]

On Wed, Jun 27, 2007 at 05:13:40PM -0400, Hal Rosenstock wrote:

> > - The kernel periodically fetches the performance stats and aggregates
> >   them into a 64 wrapping counter. The kernel sends PMA mads into the
> >   mellanox firmware to read and reset the counters
> > - The new 64 bit stats are exported via sysfs/proc/whatever as
> >   wrapping counters
> > - When a PMA packet comes in the kernel services it rather than
> >   passing it on to the chip firmware.
> 
> In this way, both 32 and 64 bit counters could be presented by the PMA
> but how would it know when the a counter has maxed out in terms of the
> PMA and how would a remote clear be handled ?

Each time the counter is cleared the kernel would store the 64 bit
value as the 'last PMA counter'. Then the calculation is just

if ((current - stored) >= saturation)
  return saturation;
return current - stored;

After 2**64 counts the saturation computation will stop working. It
would take 24 years of constant maxed out data transfer for a 12x QDR
link to wrap a 64 bit dword byte counter.

A nice side benifit would that linux drivers could present a
consistent PMA interface with new extended 64 bit counters even with
older hardware.

Jason