[openib-general] building and running IBMgtsim?

[Eitan Zahavi]

Chris Elmquist wrote:
> Hi Guys...
>
> Thank you very much for the recipe.  We actually had a success getting
> it to go just after posting to the list but these instructions will now
> confirm whether we did it the right way or not.
>
> Are there any guidelines for how big of a network the simulator can
> deal with?  Maybe something that relates it to available memory on the
> platform it is running or other resource issues?  We threw one model
> at it already which tipped it over but we are certainly not sure we are
> using it the right way yet.
>   
I was able to simulate 10K nodes in the past.
What I did to get there was to use two machines: one for the simulator 
and one for the SM.
I also used 64bit (x86_64) machines to avoid the ~3GB data limit.
> Thanks again.  We hope to be activate participants in this space going
> forward and as soon as we know what we are doing, we'll feed it back to
> the group.
>
> Chris 
>
> On Thursday (12/21/2006 at 04:59PM +0200), Eitan Zahavi wrote:
>   
>> Hi Chris,
>>
>> Sorry for my late response on this:
>>
>> The simulator is a standalone "server" where clients connect to it 
>> through a TCP/IP socket.
>>
>> OpenSM which is not built with "sim" vendor (using --with-osmv=sim 
>> --with-sim=<prefix used for ibmgtsim install>)
>> will not try to connect to the simulator but will go to the real IB 
>> network instead.
>>
>> So you need a second "simulator" install of OpenSM.
>> You can simply clone the GIT tree and
>> ./autogen.sh
>> ./configure --with-osmv=sim --with-sim=<prefix used for ibmgtsim 
>> install> --prefix=<somewhere>
>> make
>> make install
>>
>> RunSimTest -f OsmTest.sim.tcl -c OsmTest.check.tcl -t IS1-16.topo -o 
>> <somewhere>/bin/opensm
>>
>> Actually OsmTest is a test that currently fail (due to last changes in 
>> InformInfo),
>> but any other *.check.tcl/*.sim.tcl pair should work.
>>
>> Eitan
>>
>>
>> Chris Elmquist wrote:
>>     
>>> Folks,
>>>
>>> I am trying to build and run IBMgtsim so that I can explore some different
>>> topologies and system sizes.  But I am having a lot of trouble getting
>>> OpenSM to work with the simulator.
>>>
>>> I pulled down Eitan's ibutils git tree (to get the simulator) and
>>> am otherwise using the OFED 1.1 tarball for the rest of the stuff.
>>> I suspect I have a problem with OpenSM not being built correctly to use
>>> the simulator.
>>>
>>> Does anyone have a recipe on how to build and install all of these pieces
>>> (ie, openib, openSM and ibmgtsim) so that they will work together?
>>>
>>> I have been just trying to run one of the tests provided with the
>>> simulator like this:
>>>
>>> % cd ~/ibutils/ibmgtsim/tests
>>> % RunSimTest -c OsmTest.check.tcl -f OsmTest.sim.tcl -t IS1-16.topo -o 
>>> /usr/local/bin/opensm
>>>
>>> but we get this sort of output:
>>>
>>> -I- Using random seed:43204
>>> -I- Simulation directory is: /tmp/ibmgtsim.29716
>>> -I- Calling IBMgtSim -s 43204 -V 0xA3 -t 
>>> /root/ibutils/ibmgtsim/tests/IS1-16.top
>>> o -f /root/ibutils/ibmgtsim/tests/OsmTest.sim.tcl -l 
>>> /tmp/ibmgtsim.29716/sim.log
>>> -I- Simulator Ready
>>> -I- Connecting to the simulator control server:pcplod.americas.sgi.com 
>>> port:3726
>>> 5 
>>> -I- Connected to the simulator control server
>>> -I- Defined 51 guids
>>> -I- Node H-1 data: 0x0002c90000000008 {0x0002c90000000009 1} 
>>> {0x0002c9000000000a
>>> 2}
>>> -I- Starting: /usr/local/bin/opensm -g 0x0002c90000000009  ...
>>> -I- Waiting for OpenSM subnet up ...
>>> -I- OpenSM Event:ERR Dec 20 10:53:09 470415 [5100E100] -> 
>>> osm_vendor_open_port: ERR 5422: Unable to find requested CA guid 
>>> 0x2c90000000009
>>> -I- New 1 events of /tmp/ibmgtsim.29716/osm.log
>>> -I- OpenSM Event:ERR Dec 20 10:53:09 470419 [5100E100] -> osm_vendor_bind: 
>>> ERR 5
>>> 424: Unable to Open Port 0x2c90000000009
>>> -I- New 1 events of /tmp/ibmgtsim.29716/osm.log
>>> -I- OpenSM Event:ERR Dec 20 10:53:09 470422 [5100E100] -> 
>>> osm_sm_mad_ctrl_bind: ERR 3118: Vendor specific bind failed
>>> -I- New 1 events of /tmp/ibmgtsim.29716/osm.log
>>> -I- OpenSM Event:ERR Dec 20 10:53:09 470427 [5100E100] -> osm_sm_bind: ERR 
>>> 2E10:
>>> SM MAD Controller bind failed (IB_ERROR)
>>> -I- New 1 events of /tmp/ibmgtsim.29716/osm.log
>>> -I- OpenSM Event:ERR Dec 20 10:53:09 470434 [5100E100] -> 
>>> osm_sa_mad_ctrl_unbind
>>> : ERR 1A11: No previous bind
>>> -I- New 1 events of /tmp/ibmgtsim.29716/osm.log
>>>
>>> Thank you.
>>>
>>> Chris
>>> SGI Network Engineering
>>>  
>>>       
>
>