[openib-general] A few questions about IBMgtSim

[Eitan Zahavi]

Hi Sven,

> >
> > Currently there is no way to scale simulation time to real time.
> > The main reason is that the time scale is mixed:
> > * OpenSM calculation time is about the same (if you run the simulator
> > on remote node)
> 
> So this means that the internal operation of OpenSM with the simulator is
> identical to its operation with real hardware?
[EZ] Yes, if the algorithmic stage is only computational (like the routing stage) the time it takes is the sane as real hardware.
But the entire fabric setting is involving sending and receiving MADs thus odes not scale.
> 
> I have done some performance test with IBMgtSim and OpenSM running on
> separate machines and to me it looks like there is very little concurrency
> between the two processes. I.e. it looks like they spend a lot of time  waiting
> for each other. Below are some results from a few simulation runs, the
> observed CPU utilization seems quite low. I would have expected much higher
> CPU load for IBMgtSim...  Any thoughts on how this matches your experience?
[EZ] Yes - these is not much concurrency. Actually it really depends on the number of MADs you allow on the wire.
Also, one of the major limitations I run into (which made me split the processes to 2 machine) was memory availability for the 10K nodes case.

I do not see what is the drive for doing these comparisons. 
BTW: do you plan to run the OpenSM tests over the simulator?

> 
> OpenSM
> #hosts² #sw #ports elapsed¹ kernel¹ user¹ %cpu mem
>    288   36     24     585    109   99   35 410
>    512   48     32     766    144  136   36 520
>   1152   72     48    1161    218  211   36 741
> 
> IBMgtSim
> #hosts² #sw #ports elapsed¹ kernel¹ user¹ %cpu mem
>    288   36     24     586      87   221   52  92
>    512   48     32     767     109   278   50 102
>   1152   72     48    1161     169   432   51 132
> 
> ¹time in seconds
> ²organized in a 3 stage Clos
> 
> Best regards,
> Sven-Arne
> 
> --
> SAR
> ---- GnuPG public key - http://home.ifi.uio.no/~svenar/gpg.asc ----  "There are
> only 10 kinds of people in this world; those who know
>   binary and those who don't."
>                                            -- Unknown