[ofw] Programatically checking Infiniband performance counters

[Moshe Haim]

> > Looking at the ib_port_counters_t structure I see several fields.
> > 
> > The port_xmit_data looks like transferred the data in bytes divided
by
> > 4 - however, it is only 32 bit so it fills up very fast (after
16GB).
>
> Yes; so either the polling needs to account for this or use optional
> extended 64 bit counters if they are supported.

Could you please elaborate on the extended 64 bit counters? 
How can I tell if they are supported and how can I access them?

> > 2. How can I reset the counters to 0? I tried using ib_local_mad
with
> > setting the port_xmit_pkts/port_xmit_data to 0 and use mad->method =
> > IB_MAD_METHOD_SET but that doesn't seem to work...
> > I need this since I expect to run for long periods of time and even
> > the port_xmit_pkts will hit its limit.
>
> Did you set PortSelect and CounterSelect ?

I am basically using the code inside vstat_main.c in the function
vstat_get_counters - so PortSelect and CounterSelect are the same
values(portnum and 0xFF respectively), I only changed the mad_in->method
from IB_MAD_METHOD_GET to IB_MAD_METHOD_SET.
I get the following error: ib_local_mad failed with status = 42.

> > Last but not least, is there any plan to add performance counters in
> > windows for WinOF in the future?
> 
> In general, there can only be one performance manager owning a node's
> performance counters (assuming they are going to do resets as above).
> PerfMgrs come bundled in a number of different SMs. A more recent
OpenSM
> than what is current in Windows has this as do most if not all vendor
> SMs.

I have WinOF 1.0.1 installed with the OpenSM service started. 
So if I understand correctly the version that I have does not manage the
performance counters.
According to what you say I also understand that future versions of the
OpenSM will have Performance Manager that manages the performance
counters - am I correct? If so in which version of WinOF is this planned
for?

Also, when talking about bundled performance managers - do they
automatically reset the counters if they hit the limit, do I interact
with them in order to get/set values for the performance counters?

Thanks,
Moshe.

-----Original Message-----
From: Hal Rosenstock [mailto:hrosenstock at xsigo.com] 
Sent: Sunday, May 04, 2008 7:17 PM
To: Moshe Haim
Cc: Tzachi Dar; ofw at lists.openfabrics.org
Subject: RE: [ofw] Programatically checking Infiniband performance
counters

Moshe,

On Sun, 2008-05-04 at 15:46 +0300, Moshe Haim wrote:
> Thanks for the quick reply.
> 
>  
> 
> I have a few additional questions:
> 
> Looking at the ib_port_counters_t structure I see several fields.
> 
> The port_xmit_data looks like transferred the data in bytes divided by
> 4 - however, it is only 32 bit so it fills up very fast (after 16GB).

Yes; so either the polling needs to account for this or use optional
extended 64 bit counters if they are supported.

> The port_xmit_pkts looks like transferred data in 1KB packets

Just a packet count; nothing related to packet size.

>  - so it fills up more slowly:

Yes.

> 1. Can I rely on the packet size being 1KB - if not where can I find
> the packet size definition. 

No; the packets are all different sizes depending on the protocols being
used.

> 2. How can I reset the counters to 0? I tried using ib_local_mad with
> setting the port_xmit_pkts/port_xmit_data to 0 and use mad->method =
> IB_MAD_METHOD_SET but that doesn't seem to work...
> I need this since I expect to run for long periods of time and even
> the port_xmit_pkts will hit its limit.

Did you set PortSelect and CounterSelect ?

> Last but not least, is there any plan to add performance counters in
> windows for WinOF in the future?

In general, there can only be one performance manager owning a node's
performance counters (assuming they are going to do resets as above).
PerfMgrs come bundled in a number of different SMs. A more recent OpenSM
than what is current in Windows has this as do most if not all vendor
SMs.

-- Hal

> Thanks,
> 
> Moshe.
> 
>  
> 
>                                    
> ______________________________________________________________________
> 
> From: Tzachi Dar [mailto:tzachid at mellanox.co.il] 
> Sent: Sunday, April 27, 2008 5:51 PM
> To: Moshe Haim; ofw at lists.openfabrics.org
> Subject: RE: [ofw] Programatically checking Infiniband performance
> counters
> 
> 
>  
> 
> Using performance counters currently only allows one to get WSD data.
> Since you are working on XP that won't help you. You can use the same
> way that vstat is doing to get that data.
> 
> 
> In other words, you need to implement the function vstat_get_counters
> () that is using a local mad in order to get that data.
> 
> 
>  
> 
> 
> You can find this function in the file vstat_main.c (lines 343 - 393)
> 
> 
>  
> 
> 
> Thanks
> 
> 
> Tzachi
> 
> 
>          
>         
>                                        
>         ______________________________________________________________
>         
>         From:ofw-bounces at lists.openfabrics.org [mailto:ofw-
>         bounces at lists.openfabrics.org] On Behalf Of Moshe Haim
>         Sent: Sunday, April 27, 2008 5:09 PM
>         To: ofw at lists.openfabrics.org
>         Subject: [ofw] Programatically checking Infiniband performance
>         counters
>         
>         Hi,
>         
>          
>         
>         I am working with Infiniband on Windows XP using the
>         WinOF_1.0.1 package.
>         
>         I need to be able to check the Infiniband performance counters
>         in order to estimate traffic over the Infiniband.
>         
>          
>         
>         I have seen that using vstat -c the counter information
>         appears.
>         
>         How can I retrieve it not using vstat.exe? Are there any
>         performance counters I can check (for instance select them in
>         perfmon.exe?), or any other APIs?
>         
>          
>         
>         Thanks,
>         
>         Moshe.
>         
>          
>         
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw