[ofa-general] Running OpenSM on large clusters

[Sasha Khapyorsky]

On 13:40 Wed 17 Oct     , Maestas, Christopher Daniel wrote:
> We had some similar experiences at 4480 nodes using Open SM 3.0.0 svn
> tag 10188.

It is something pre-OFED-1.2. 

> With a clean mapping I think it took ~3-5 minutes.

There were many performance improvements since that (but probably my
simulator is too fast anyway :)).

BTW could you send me output of ibnetdiscover? I will be able to re-run
it with ibsim too.

Sasha

> 
> -----Original Message-----
> From: general-bounces at lists.openfabrics.org
> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Sasha
> Khapyorsky
> Sent: Wednesday, October 17, 2007 1:24 PM
> To: Ira Weiny
> Cc: Edward Mascarenhas; general at lists.openfabrics.org
> Subject: Re: [ofa-general] Running OpenSM on large clusters
> 
> On 21:03 Wed 17 Oct     , Sasha Khapyorsky wrote:
> > > 
> > > We have atlas running with 1152 nodes.  OpenSM is able to route it 
> > > with up/down routing in ~2min.
> > 
> > 2min is a lot for OpenSM with up/down. Is it pure OpenSM time or from 
> > bring-up power-on?
> 
> With simulator (ibsim) and atlas I have 7+ seconds with master OpenSM:
> 
> -------------------------------------------------
> OpenSM 3.1.5
> Command Line Arguments:
>  Creating new log file
>  Run Once
>  Log File: ./osm.log
> -------------------------------------------------
> OpenSM 3.1.5
> 
> Using default GUID 0x2c9020021a5ed
> Entering MASTER state
> 
> SUBNET UP
> 
> Exiting SM
> 
> 
> real	0m7.324s
> user	0m5.860s
> sys	0m2.980s
> 
> 
> Sasha
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
> 
>