<div dir="ltr">I just realized that in this example I'm shutting down the entire switch that the host is connected to instead of the uplink port. If I issue 'clear "S-0002c90300684e30" 2"', I get the same result. Port 1 and 2 are both uplink ports to different leaf IB switches in a fat tree scheme.</div>
<div class="gmail_extra"><br clear="all"><div><div><span style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"><br></span></div><span style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">Robert LeBlanc</span><br style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">
<span style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">OIT Infrastructure & Virtualization Engineer</span><br style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">
<span style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">Brigham Young University</span></div>
<br><br><div class="gmail_quote">On Fri, Oct 25, 2013 at 11:19 AM, Robert LeBlanc <span dir="ltr"><<a href="mailto:robert_leblanc@byu.edu" target="_blank">robert_leblanc@byu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Here is the details of what I'm doing:<div><br></div><div>In one terminal, I run ibsim:</div><div><div><font face="courier new, monospace">root@rleblanc-pc:/home/leblanc/Downloads# ibsim -s ibtopo </font></div>
<div><font face="courier new, monospace">parsing: ibtopo</font></div><div><font face="courier new, monospace">ibtopo: parsed 928 lines</font></div><div><font face="courier new, monospace">########################</font></div>
<div><font face="courier new, monospace">Network simulator ready.</font></div><div><font face="courier new, monospace">MaxNetNodes = 2048</font></div><div><font face="courier new, monospace">MaxNetSwitches = 256</font></div>
<div><font face="courier new, monospace">MaxNetPorts = 13312</font></div><div><font face="courier new, monospace">MaxLinearCap = 30720</font></div><div><font face="courier new, monospace">MaxMcastCap = 1024</font></div>
<div><font face="courier new, monospace">sim> ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div><div><font face="courier new, monospace">ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div>
<div><font face="courier new, monospace">...snip out tons of these messages...</font></div><div><font face="courier new, monospace">ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div>
<div><font face="courier new, monospace">clear "S-0002c90300684e30"</font></div><div><font face="courier new, monospace">sim> ibwarn: [2278] process_packet: got trap repress - drop</font></div><div><font face="courier new, monospace">ibwarn: [2278] process_packet: got trap repress - drop</font></div>
<div><font face="courier new, monospace">ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div><div><font face="courier new, monospace">...snip out tons of these messages...</font></div>
<div><font face="courier new, monospace">ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div><div><font face="courier new, monospace">relink "0002c90300684e30"</font></div><div>
<font face="courier new, monospace"># nodeid "0002c90300684e30" (0002c90300684e30) not found</font></div><div><font face="courier new, monospace">sim> relink "S-0002c90300684e30"</font></div><div><font face="courier new, monospace">sim> ibwarn: [2278] process_packet: got trap repress - drop</font></div>
<div><font face="courier new, monospace">ibwarn: [2278] process_packet: got trap repress - drop</font></div><div><font face="courier new, monospace">ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div>
<div><font face="courier new, monospace">...snip out tons of these messages...</font></div><div><font face="courier new, monospace">ibwarn: [2278] process_packet: no one to handle pkt: class 0x81, attr 0xff90</font></div>
<div><font face="courier new, monospace">quit</font></div><div><font face="courier new, monospace">Exiting network simulator.</font></div><div><font face="courier new, monospace">root@rleblanc-pc:/home/leblanc/Downloads# </font></div>
<div><br></div><div>Then in another terminal I run opensm:</div><div><div><font face="courier new, monospace">root@rleblanc-pc:/home/leblanc/Documents/Work/Scripts/ib# SIM_HOST="H-0013970201000978" OSM_TMP_DIR=./ OSM_CACHE_DIR=./ LD_PRELOAD=/usr/lib/umad2sim/libumad2sim.so opensm -e -v -f ./osm.log </font></div>
<div><font face="courier new, monospace">-------------------------------------------------</font></div><div><font face="courier new, monospace">OpenSM 3.3.15</font></div><div><font face="courier new, monospace">Command Line Arguments:</font></div>
<div><font face="courier new, monospace"> Creating new log file</font></div><div><font face="courier new, monospace"> Verbose option -v (log flags = 0x7)</font></div><div><font face="courier new, monospace"> Log File: ./osm.log</font></div>
<div><font face="courier new, monospace">-------------------------------------------------</font></div><div><font face="courier new, monospace">OpenSM 3.3.15</font></div><div><font face="courier new, monospace"><br></font></div>
<div><font face="courier new, monospace">Entering DISCOVERING state</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">Using default GUID 0x13970201000979</font></div>
<div><font face="courier new, monospace">Entering MASTER state</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">=======================================================================================================</font></div>
<div><font face="courier new, monospace">Vendor : Ty : # : Sta : LID : LMC : MTU : LWA : LSA : Port GUID : Neighbor Port (Port #)</font></div><div><font face="courier new, monospace">Unknown : CA : 01 : ACT : 0003 : 0 : 2048 : 4x : 2.5 : f04da29097793001 : 0002c9020042ea60 (12)</font></div>
<div><font face="courier new, monospace">Unknown : CA : 02 : ACT : 0007 : 0 : 2048 : 4x : 2.5 : f04da29097793002 : 0002c902004294e0 (12)</font></div><div><font face="courier new, monospace">------------------------------------------------------------------------------------------------------</font></div>
<div><font face="courier new, monospace">Mellanox : SW : 00 : : 0002 : 0 : : : : 0002c90300879a00 :</font></div><div><font face="courier new, monospace">Mellanox : SW : 01 : ACT : : : 2048 : 4x : 2.5 : 0002c90300879a00 : 0002c90200431f90 (08)</font></div>
<div><font face="courier new, monospace">Mellanox : SW : 02 : ACT : : : 2048 : 4x : 2.5 : 0002c90300879a00 : 0002c90200431f58 (09)</font></div><div><font face="courier new, monospace">Mellanox : SW : 03 : DWN : : : ??? : ??? : Ext : 0002c90300879a00 :</font></div>
</div><div><font face="courier new, monospace">...snip...</font></div><div><br></div><div>Then in a third console I run ibtracert:</div><div><div><font face="courier new, monospace">leblanc@rleblanc-pc:~/Documents/Work/Scripts/ib$ LD_PRELOAD=/usr/lib/umad2sim/libumad2sim.so /usr/sbin/ibtracert -G -n 0x0002c90300ebbb62 0x0013970301001f4c 2> /dev/null </font></div>
<div><font face="courier new, monospace">From {0x0002c90300ebbb60}[2]</font></div><div><font face="courier new, monospace">[2] -> {0x0002c90300684e30}[19]</font></div><div><font face="courier new, monospace">[2] -> {0x0002c90200431eb8}[10]</font></div>
<div><font face="courier new, monospace">[33] -> {0x001397010a000044}[10]</font></div><div><font face="courier new, monospace">[35] -> {0x0013970301001f4c}[1]</font></div><div><font face="courier new, monospace">To {0x0013970301001f4b}[1]</font></div>
<div><font face="courier new, monospace">leblanc@rleblanc-pc:~/Documents/Work/Scripts/ib$ LD_PRELOAD=/usr/lib/umad2sim/libumad2sim.so /usr/sbin/ibtracert -G -n 0x0002c90300ebbb62 0x0013970301001f4c 2> /dev/null </font></div>
<div><font face="courier new, monospace">/usr/sbin/ibtracert: iberror: failed: can't resolve source port 0x0002c90300ebbb62</font></div><div><font face="courier new, monospace">leblanc@rleblanc-pc:~/Documents/Work/Scripts/ib$ LD_PRELOAD=/usr/lib/umad2sim/libumad2sim.so /usr/sbin/ibtracert -G -n 0x0002c90300ebbb62 0x0013970301001f4c 2> /dev/null </font></div>
<div><font face="courier new, monospace">From {0x0002c90300ebbb60}[2]</font></div><div><font face="courier new, monospace">[2] -> {0x0002c90300684e30}[19]</font></div><div><font face="courier new, monospace">[2] -> {0x0002c90200431eb8}[10]</font></div>
<div><font face="courier new, monospace">[33] -> {0x001397010a000044}[10]</font></div><div><font face="courier new, monospace">[35] -> {0x0013970301001f4c}[1]</font></div><div><font face="courier new, monospace">To {0x0013970301001f4b}[1]</font></div>
<div><font face="courier new, monospace">leblanc@rleblanc-pc:~/Documents/Work/Scripts/ib$ </font></div><div><br></div></div></div><div>I'm attaching our topo file that we are using and the opensm logs (you should be able to replicate the problem given this information or tell me what I'm doing wrong).</div>
<div><br></div><div>Thanks,</div><div class="gmail_extra"><div class="im"><br clear="all"><div><div><span style="font-size:13px;font-family:arial,sans-serif"><br></span></div><span style="font-size:13px;font-family:arial,sans-serif">Robert LeBlanc</span><br style="font-size:13px;font-family:arial,sans-serif">
<span style="font-size:13px;font-family:arial,sans-serif">OIT Infrastructure & Virtualization Engineer</span><br style="font-size:13px;font-family:arial,sans-serif">
<span style="font-size:13px;font-family:arial,sans-serif">Brigham Young University</span></div>
<br><br></div><div><div class="h5"><div class="gmail_quote">On Tue, Oct 22, 2013 at 10:55 PM, Hal Rosenstock <span dir="ltr"><<a href="mailto:hal.rosenstock@gmail.com" target="_blank">hal.rosenstock@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>ibsim just simulates the network (topology, SMAs, and PMAs). OpenSM configured the subnet including the routing (LFTs and MFTs) based on the routing algorithm. It is possible in a topology that multiple routing algorithms yield the same routes. More specifics would be needed to comment "deeper"...</div>
<div> </div><div>-- Hal</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Tue, Oct 22, 2013 at 6:38 PM, Robert LeBlanc <span dir="ltr"><<a href="mailto:robert_leblanc@byu.edu" target="_blank">robert_leblanc@byu.edu</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">I'm trying to test routing in ibsim, but it doesn't seem to update the routing tables in the simulated switches. If I take a link down using the clear command in ibsim, I see opensm saying that it is updating the routing tables and that it completes, but I can't ibtracert to the LID who's path was taken down.<div>
<br></div><div>Should ibsim and opensm be reconfiguring routing in the simulated environment? No matter which routing protocol I select in opensm, the routes are always the same, even having opensm re-LID the entire fabric doesn't help. Any help would be appreciated.</div>
<div><br></div><div>Output from opensm:</div><div><br></div><div><div><font face="courier new, monospace">******************************************************************</font></div><div><font face="courier new, monospace">***** LID ASSIGNMENT COMPLETE - STARTING SWITCH TABLE CONFIG *****</font></div>
<div><font face="courier new, monospace">******************************************************************</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace">Oct 22 16:27:20 330198 [8437A700] 0x04 -> osm_ucast_mgr_build_lid_matrices: Starting switches' Min Hop Table Assignment</font></div><div><font face="courier new, monospace">Oct 22 16:27:20 330954 [8437A700] 0x02 -> osm_ucast_mgr_process: minhop tables configured on all switches</font></div>
<div><font face="courier new, monospace">Oct 22 16:27:20 331191 [8437A700] 0x04 -> do_sweep:</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><br></font></div>
<div><font face="courier new, monospace">******************************************************************</font></div><div><font face="courier new, monospace">**************** SWITCHES CONFIGURED FOR UNICAST *****************</font></div>
<div><font face="courier new, monospace">******************************************************************</font></div></div><div><br></div><div><br></div><div>Thanks,<br clear="all"><div><div><span style="font-family:arial,sans-serif;font-size:13px"><br>
</span></div><span style="font-family:arial,sans-serif;font-size:13px">Robert LeBlanc</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">OIT Infrastructure & Virtualization Engineer</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">Brigham Young University</span></div>
</div></div>
<br></div></div>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.openfabrics.org" target="_blank">Users@lists.openfabrics.org</a><br>
<a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/users" target="_blank">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/users</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>