[ofa-general] Obtaining RDMA statistics for an InfiniBand interface

[Hal Rosenstock]

On Mon, 2008-05-12 at 21:05 +0200, Pawel Dziekonski wrote:
> On Mon, 12 May 2008 at 12:00:13PM -0700, Hal Rosenstock wrote:
> > On Mon, 2008-05-12 at 10:54 +0200, Pawel Dziekonski wrote:
> > > On Tue, 18 Mar 2008 at 05:53:06AM -0700, Hal Rosenstock wrote:
> > > > On Tue, 2008-03-18 at 09:17 +0100, Bart Van Assche wrote:
> > > > > On Mon, Mar 17, 2008 at 6:07 PM, Boris Shpolyansky <boris at mellanox.com> wrote:
> > > > > > Check :
> > > > > >
> > > > > >  /sys/class/infiniband/[mlx4_*|mthca*]/ports/*/counters/*
> > > > > 
> > > > > Thanks, these are interesting counters. Unfortunately these counters
> > > > > are 32-bit counters and already overflowed during the test I ran (less
> > > > > than one day of SRP communication):
> > > > > 
> > > > > $ uname -r
> > > > > 2.6.24
> > > > > $ uname -m
> > > > > x86_64
> > > > > $ head /sys/class/infiniband/mthca0/ports/1/counters/*_{data,packets}
> > > > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_rcv_data <==
> > > > > 4294967295
> > > > > 
> > > > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_data <==
> > > > > 4294967295
> > > > > 
> > > > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_rcv_packets <==
> > > > > 4294967295
> > > > > 
> > > > > ==> /sys/class/infiniband/mthca0/ports/1/counters/port_xmit_packets <==
> > > > > 4294967295
> > > > 
> > > > Depending on which fabric management (usually bundled with the SM) is
> > > > being used, you may be able to obtain this information via the
> > > > Performance Manager (and not be limited to 32 bit counters).
> > > 
> > > Hi,
> > > 
> > > I have exactly the same problem on OFED 1.2.5.5, redhat kernel
> > > 2.6.9-55.0.12.ELsmp on X86_64 machines. {data,packets} files contain
> > > 4294967295 value, or barely change. :(
> > > 
> > > How can I get Performance Manager running and printing some reasonable
> > > numbers?
> > 
> > Are you using OpenSM ? If so, which version ?
> 
> yes, opensm-3.0.3.

3.0.3 or 3.0.13 ? Anyway, PerfMgr was not part of those 3.0.x versions.
I think it was added in the 3.1 series (and is available in OFED 1.3) or
3.2 series (trunk). Is upgrading a possibility ? If so, you might want
to check out Ira's response on how to do this (and also what the output
looks like to make sure it can meet your needs).

-- Hal

> P