[openib-general] A few questions about IBMgtSim

[Eitan Zahavi]

Hi Sven,

Please see my response below:

Eitan

Sven-Arne Reinemo wrote:
> Hi,
> 
> After some testing of IBMgtSim I have a few questions:
> 
> 1) If I try to build topologies using the MTS14400.ibnl as a building
> block my simulation fails with a "child process exited abnormally"
> message. I guess this is related to ibdmchk since the ibdmchk log
> contains lots of errors like the following:
> 
> -I- Tracing all CA to CA paths for Credit Loops potential ...
> -E- Potential Credit Loop on Path from:H-1/U1/1 to:H-11/U1/1
>   Going:Down from:node:0002c9000000007d to:node:0002c9000000006a
>   Going:Up from:node:0002c9000000006a to:node:0002c90000000076
This error indicate what it say: The resulting routing has a potential credit
loop as it does not follow an up/down routing scheme. Credit loops can really
generated by the OpenSM on some topologies and can be avoided by adding the
-R updn flag. And possible also --add_guid_file if the SM is not able to
recognize the root nodes automatically (if the topology is highly not symmetric).

> 
> -I- Generating non blocking full link coverage plan
> into:/tmp/ibdmchk.non_block_
> all_links
> -E- After 32 stages some switch ports are still not covered:
> -E- Fail to cover port:system:0002c90000000054/node:0002c90000000054/P15
This means that there is no route that goes through that port.
I.e. if you trace from all HCA to all other HCA you never go through that port.
> 
> I have included two topology files. One that works and one that fails,
> the only difference is that the number of hosts are increased from 18 to
> 20. Also, if I create my own simple ibnl file for a switch with 144 (or
> other sizes) ports I am able to run simulations. Any suggestions to what
> the problem might be?
As described above the reason is credit loop potential and the specific topology
and routing algorithm used. Please try the -R updn and --add_guid_file.
You can scan the ibmgtsim.guids.txt file to know the GUIDS assigned to the spine switches.
> 
> 
> 2) The included example ibmgtsim/tests/RhinoBased10K.topo never finishes
> (at least not in 24 hours). Does this work for anyone else? All other
> examples work fine.
I was able to simulate it by:
1. Decreasing the verbosity
2. Running the simulator on one machine and the OpenSM on another
> 
> 3) If I would like to use IBMgtSim with my own (simplified) SM would it
> be straightforward? It looks too me like RunSimTest talks to any SM
> given the correct path, node and port number for location of the SM.
You can use libibmscli.so/.a to integrate your SM with ibmgtsim.
This lib API is provided in ibms_client_api.h
It mainly enables connecting to the ibmgtsim server TCP/IP port declaring
the port the SM is attached to, registering to receive some MAD class/attributes
sending and  receiving MADs.

>