<br><br><div class="gmail_quote">On Fri, Jun 7, 2013 at 4:35 PM, Orion Poplawski <span dir="ltr"><<a href="mailto:orion@cora.nwra.com" target="_blank">orion@cora.nwra.com</a>></span> wrote:<br><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<div class="im">On 06/07/2013 02:23 PM, Hal Rosenstock wrote:<br>
<br>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
Looks like your 2 subnets are "interconnected" so they're not really 2<br>
disjoint subnets! Is your other subnet 0xfe80::5 ? Looking at your<br>
ibnetdiscover file, there's only 1 switch so are you running 2 SMs<br>
(one for<br>
each subnet) over the same topology. If so, that doesn't work.<br>
<br>
<br>
I should only have 2 subnets, and we should only be seeing the 0xfe80::1<br>
subnet here (there is a 0xfe80::2 subnet that consist only of two machines<br>
(amos and andrew) directly connected together). With the MT25204 windows<br>
machine, 5:ad00:c:5ced is the GUID I believe, so it looks like it may have<br>
a prefix of 0xfe80::0 ? I confirmed that the SM service on the windows<br>
machine (fontdb) is disabled and stopped. So I have no idea why it isn't<br>
getting a prefix of 0xfe80::1.<br>
<br>
Yes, I see now. It does have the default subnet prefix rather than the one you<br>
configured in the SM. This is evidence of what you asked before which is why<br>
you probably asked. I don't know whether or not non default subnet prefixes<br>
work on Windows. Is there any reason you want to run this with other than the<br>
default subnet prefix ? If not, can you try that and see if things work ?<br>
While it is legal to have different IB subnets on the same IPoIB subnet, that<br>
requires an IB router and isn't your intent anyway.<br>
</blockquote>
<br>
<br></div>
This is one reason I'm running with a non-default subnet ID:<br>
<br>
<a href="http://www.open-mpi.org/faq/?category=openfabrics#ofa-default-subnet-gid" target="_blank">http://www.open-mpi.org/faq/?<u></u>category=openfabrics#ofa-<u></u>default-subnet-gid</a><br>
<br>
and I do have some multi-homed machines (amos and andrew above) and may add some more.<div class="im"><br>
<br></div></blockquote><div> </div><div>Can you run the default prefix on the problematic subnet and another non default one on the other back to back one (at least to see if it works or not) ?</div><div> </div><div> </div>
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote"><div class="im">
<blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
Also, if you turn on log verbosity on OpenSM temporarily and send me the log<br>
for that run, I could see what is going on with in terms of trying to set the<br>
non default subnet prefix with the Windows node. Given the log you sent, I can<br>
only imagine that the SMA on the Windows node is ack'ing the PortInfo set<br>
which sets the subnet prefix but not really acting on it properly.<br>
-- Hal<br>
</blockquote>
<br></div>
There are a lot of different levels for verbosity. What would be useful (but perhaps not too much)?<br></blockquote><div> </div><div>I would just go for 0xFF and get the too much version for now for the purposes of debug and then switch it back.</div>
<div> </div><div>-- Hal</div><div> </div><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<br>
Thanks!<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
-- <br>
Orion Poplawski<br>
Technical Manager <a href="tel:303-415-9701%20x222" target="_blank" value="+13034159701">303-415-9701 x222</a><br>
NWRA, Boulder/CoRA Office FAX: <a href="tel:303-415-9702" target="_blank" value="+13034159702">303-415-9702</a><br>
3380 Mitchell Lane <a href="mailto:orion@nwra.com" target="_blank">orion@nwra.com</a><br>
Boulder, CO 80301 <a href="http://www.nwra.com" target="_blank">http://www.nwra.com</a><br>
</div></div></blockquote></div><br>