[openib-general] [RFC] Performance Manager

[Hal Rosenstock]

On Fri, 2007-01-26 at 18:15, Sean Hefty wrote:
> >There are numerous PerfManager models which can be supported:
> >1. Integrated as thread(s) with OpenSM (run only when SM is master)
> >2. Standby SM
> >3. Standalone PerfManager (not running with master or standby SM)
> >4. Distributed PerfManager (most scalable approach)
> 
> IMO, we will eventually need distributed managers, 

Yes.

> so I would go with the last approach.

Initially ? It is also an implementation phasing issue as stated. The
core support is needed in both so there is very little unneeded work to
get to the first phase in terms of a distributed approach. We would
certainly grow/evolve towards this after that initial implementation.

> But, along those lines, if we had a distributed SM,

There has been some early discussion on a distributed SA. Distributing
SM is much harder IMO.

> would you still want to separate the performance manager from the SM?  
> It seems more flexible, but with additional load on the fabric.

Ideally, it would be a deployment choice and the implementation would
support both modes. The problem is that we've already seen that the SM
node has enough to do at times in a large cluster and PerfManagement in
addition with its constant demands is likely not a good addition in
terms of this.

The additional fabric load is twofold: first, the reports for nodes
coming and going, and second, any intermanager communication. I don't
think the first is a significant load and I'm not yet sure about the
second. In any case, the second load can be constrained to the portion
of the subnet where the management nodes are in those cases where this
is a concern.

> >In terms of inter manager communication, there seem to be several
> >choices:
> >1. Use vendor specific MADs (which can be RMPP'd) and build on top of
> >this
> >2. Use IPoIB which is much more powerful as sockets can then be utilized.
> 
> You could also use RC QP communication up/down the hierarchy.

Wouldn't that have the same issues as approach 1 (as compared with
approach 2) ?

-- Hal

> - Sean