Fwd: [ofa-general] Performance evaluation of Opensm

Tue Jul 7 04:57:57 PDT 2009

Devesh Sharma wrote:
> Thanks Yevgeny, for your valuable input. This will surly help for my work.
> 
> On Tue, Jul 7, 2009 at 2:59 PM, Yevgeny
> Kliteynik<kliteyn at dev.mellanox.co.il> wrote:
>> Hi Davesh,
>>
>> It's kind of hard to talk about "performance of OpenSM".
>> Subnet Manager has different phases and modes of operation,
>> each of them is completely separate issue:
>>
>> - Fabric discovery
>> - Fabric ports/nodes configuration
>> - Unicast routing calculation
>> - Unicast routing configuration on fabric switches
>> - Multicast routing calculation
>> - Multicast routing configuration on fabric switches
>> - SA queries processing
>> - Memory consumption
>> - Different routing algorithms consume different time and memory
>> - QoS
>> - etc, etc, etc
>>
>> Most of the above can be measured only on real cluster.
>
> But how these can be measured is there any compile time flag available
> in the Code?

You can run SM with higher verbosity, and then you will see more
log messages in the SM log file (run 'opensm -h', or 'man opensm'
to see its usage). Each log message has time stamp.
Running 'opensm -D 7' will allow you to see SM state transitions.
You can bump up the verbosity even more to see more details.
However, note that high verbosity adds to the OSM runtime, so
the best way to measure certain SM operations would be an old 
fashion way - add messages into the SM code and run SM with the
usual verbosity.

>> Some (such as routing calculation and memory consumption) can
>> be measured while OSM is running on top of the simulator.
>
> Simulation results are far far away from real situation..:( I am
> interested in results with the real fabric.

Agree.

>> Some are very affected by the number of CPU cores that you
>> have on the management node (e.g. SA queries processing),
>> others mostly affected by the CPU frequency (unicast routing).
>> Also, various OpenSM options can affect these phases, such as
>> unicast routing cache may reduce routing calculation time to 0.
> Hmm........correct.
>> Sorry that I'm not really answering your question :(
>> I just want to point out the fact that there are many aspects
>> that should be considered when talking about OpenSM performance.
>
> Do we have any such tool with does profiling of all these phases of
> SM. Such tool will be
> helpful for the researcher working on different algorithms related to SM.

No tool (at least not that I'm aware of).
Just examining the log, and/or adding log messages/printf's to
the OpenSM code.

>> If what you're interested in is just "system-wide" numbers,
>> then you'll probably want to know how much time it takes for
>> the OpenSM to bring up cluster from scratch, or how much time
>> it takes to reconfigure the fabric after some change.
>
> Will it be fine if I run OpenSM with "time" command and press Ctrl-C
> moment I see
> SUBNET UP msg. Of-course keeping some of the options and
> configurations as constant?
> 
> # time opensm -<some options>
> SUBNET UP Ctrl-C

Sure, it's possible.
You can run it as follows: 'time opensm -o'
It will cause SM to run once and exit, so no need for ^C.
It will add some overhead of cleaning up, but it's a small overhead.
You can also run 'opensm -e -d2 &', and then examine OSM log - 
look for the "SUBNET UP" message in the log. This way you will
get rid of the clean-up overhead.

Check OpenSM usage - it might give you some more ideas.

-- Yevgeny

>> I think that these numbers can come only from the guys that
>> administer IB clusters, and they would be highly dependent
>> on the management node CPU.
>>
>> -- Yevgeny
>>
>> dhananjay tembe wrote:
>>> Hi Devesh,
>>>
>>>       OFED installer comes with a tool called as "osmtest" which is a test
>>> program for opensm and for subnet administrator that comes within the opensm
>>> itself. The command is /usr/sbin/osmtest.
>>>
>>>       Another way to test it might be to try it in a large fabric. If you
>>> do not have a large fabric, try using the simulator that comes with ofed.
>>> The command for the simulator is /usr/bin/ibsim
>>> I have not used the simulator yet. So I do not know much about it.
>>>
>>> Thanks and regards,
>>> ---Dhananjay.
>>>
>>>
>>> --- On Tue, 7/7/09, Devesh Sharma <devesh28 at gmail.com> wrote:
>>>
>>>> From: Devesh Sharma <devesh28 at gmail.com>
>>>> Subject: Fwd: [ofa-general] Performance evaluation of Opensm
>>>> To: general at lists.openfabrics.org
>>>> Date: Tuesday, 7 July, 2009, 11:13 AM
>>>> On Mon, Jul 6, 2009 at 8:33 PM, Peter
>>>> Kjellstrom<cap at nsc.liu.se>
>>>> wrote:
>>>>> On Monday 06 July 2009, Devesh Sharma wrote:
>>>>>> Hello list.
>>>>>>
>>>>>> Is there any tool or method available to get some
>>>> performance numbers
>>>>>> related to Opensm? What are the major performance
>>>> parameters of OpenSM
>>>>>> which can
>>>>>> be used for performance measurement analaysis?
>>>>> Do you refer to the performance of the fabric as setup
>>>> by OpenSM or the
>>>>> performance of OpenSM itself?
>>>> Performance of OpenSM itself. There are many tool available
>>>> to measure
>>>> performance of Fabric.
>>>>> /Peter
>>>>>
>>>>>> Thanks in advance
>>>>>> regards
>>>>>> Devesh Sharma
>>>>> _______________________________________________
>>>>> general mailing list
>>>>> general at lists.openfabrics.org
>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>>>
>>>>> To unsubscribe, please visit
>>>>> http://openib.org/mailman/listinfo/openib-general
>>>>>
>>>> _______________________________________________
>>>> general mailing list
>>>> general at lists.openfabrics.org
>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>>
>>>> To unsubscribe, please visit
>>>> http://openib.org/mailman/listinfo/openib-general
>>>>
>>>
>>>      Looking for local information? Find it on Yahoo! Local
>>> http://in.local.yahoo.com/
>>> _______________________________________________
>>> general mailing list
>>> general at lists.openfabrics.org
>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>>
>>> To unsubscribe, please visit
>>> http://openib.org/mailman/listinfo/openib-general
>>>
>>
> 