<br><br><div class="gmail_quote">On Fri, Jun 7, 2013 at 8:55 PM, Hal Rosenstock <span dir="ltr"><<a href="mailto:hal.rosenstock@gmail.com" target="_blank">hal.rosenstock@gmail.com</a>></span> wrote:<br><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<br><br><div class="gmail_quote"><div class="im">On Fri, Jun 7, 2013 at 6:52 PM, Orion Poplawski <span dir="ltr"><<a href="mailto:orion@cora.nwra.com" target="_blank">orion@cora.nwra.com</a>></span> wrote:<br><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<div>On 06/07/2013 02:23 PM, Hal Rosenstock wrote:<br>
<br>
</div><div><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
Also, if you turn on log verbosity on OpenSM temporarily and send me the log<br>
for that run, I could see what is going on with in terms of trying to set the<br>
non default subnet prefix with the Windows node. Given the log you sent, I can<br>
only imagine that the SMA on the Windows node is ack'ing the PortInfo set<br>
which sets the subnet prefix but not really acting on it properly.<br>
-- Hal<br>
</blockquote>
<br></div>
Full log is at <a href="http://sw.cora.nwra.com/test/opensm.debug.log.gz" target="_blank">http://sw.cora.nwra.com/test/<u></u>opensm.debug.log.gz</a><br>
<br></blockquote><div> </div></div><div>Looking at that log, I didn't see _any_ MC joins from that port (GUID 0x5ad00000c5ced) so this is a different scenario than before :-(</div><div> </div><div>Also, the previous confusion with:</div>
<div> </div><div># saquery -m 0xc000<br> PortGid.................fe80::<u></u>1:5:ad00:c:5c3d (Topspin DDR-HCAe LX x8)<div><br> PortGid.................fe80::<u></u>1:19:bbff:ff00:5851 (saga mthca0)<br>
</div> PortGid.................fe80::<u></u>1:19:bbff:ff00:3899 (sfcomp1 mthca0)<div><br> PortGid.................fe80::<u></u>1:1a:4bff:ff0c:20c9 (HP Lion Cub 128MB)<br> <font color="#ff0000">PortGid.................fe80::<u></u>5:ad00:c:5ced (MT25204 InfiniHostLx Mellanox Technologies)</font><br>
PortGid.................fe80::<u></u>1:17:8ff:ffd0:9df9 (alexandria2 HCA-1)<br></div><div>GUID is <font color="#ff0000">5:ad00:c:5ced</font> and prefix is <font color="#ff0000">fe80::<u></u></font> so it's either missing a digit like 1 (fe80::1 like the others) or if it's a 0 it would have a 3rd colon (fe80:::). So I'm not sure what's going on there either.</div>
<div> </div></div></div></blockquote><div>I did find a half world PR query from that node though and it's GID looks similar so I must be mistaken about the extra colon but the bottom line is same that the prefix is set in SMA but not being used in the SA queries (PR and MCM) issued by the Windows node.</div>
<div><font size="3" face="Times New Roman"></font> </div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3">Jun 07 14:56:58 353610 [11BC4700] 0x20 ->
SA MAD dump:</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>base_ver................0x1</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>mgmt_class..............0x3</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>class_ver...............0x2</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>method..................0x12
(SubnAdmGetTable)</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>status..................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>resv....................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>trans_id................0x100000002</font></span></p><div>
<font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>attr_id.................0x35
(PathRecord)</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>resv1...................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>attr_mod................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>rmpp_version............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>rmpp_type...............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>rmpp_flags..............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>rmpp_status.............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>seg_num.................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>payload_len/new_win.....0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>sm_key..................0x0000000000000000</font></span></p>
<div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>attr_offset.............0x8</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>resv2...................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>comp_mask...............0x0000000000003008</font></span></p>
<div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"> </font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3">Jun 07 14:56:58 353743 [1A7D2700] 0x08 ->
PathRecord dump:</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>service_id..............0x0000000000000000</font></span></p>
<div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>dgid....................::</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>sgid....................fe80::5:ad00:c:5ced</font></span></p>
<div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>dlid....................0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>slid....................0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>hop_flow_raw............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>tclass..................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>num_path_revers.........0xFF</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>pkey....................0xFFFF</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>qos_class...............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>sl......................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>mtu.....................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>rate....................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>pkt_life................0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>preference..............0x0</font></span></p><div><font size="3" face="Times New Roman">
</font></div><p style="margin:0in 0in 0pt" class="MsoNormal"><span style="font-family:"Courier New""><font size="3"><span style> </span>resv2...................0x000000000000</font></span></p>
<div><font size="3" face="Times New Roman">
</font></div><div> </div><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote"><div class="gmail_quote"><div><div>
</div>
<div> </div></div><div><div class="h5"><div> </div><div> </div><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
I had fontdb shutdown when I started opensm - then booted it up.<br> </blockquote><blockquote style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid" class="gmail_quote">
<br>
This seems to be when it first comes up (lid 0, prefix 0xfe80::0)<br>
<br>
Jun 07 14:56:58 088453 [193D0700] 0x10 -> osm_pi_rcv_process: [<br>
Jun 07 14:56:58 088465 [193D0700] 0x08 -> PortInfo dump:<br>
port number..............1<br>
node_guid................<u></u>0x0005ad00000c5cec<br>
port_guid................<u></u>0x0005ad00000c5ced<br>
m_key....................<u></u>0x0000000000000000<br>
subnet_prefix............<u></u>0xfe80000000000000<br>
base_lid.................0<br>
master_sm_base_lid.......0<br>
capability_mask..........<u></u>0x2500A68<br>
diag_code................0x0<br>
m_key_lease_period.......0x0<br>
local_port_num...........1<br>
link_width_enabled.......0x3<br>
link_width_supported.....0x3<br>
link_width_active........0x2<br>
link_speed_supported.....0x3<br>
port_state...............INIT<br>
state_info2..............0x52<br>
m_key_protect_bits.......0x0<br>
lmc......................0x0<br>
link_speed...............0x13<br>
mtu_smsl.................0x20<br>
vl_cap_init_type.........0x30<br>
vl_high_limit............0x0<br>
vl_arb_high_cap..........0x8<br>
vl_arb_low_cap...........0x8<br>
init_rep_mtu_cap.........0x4<br>
vl_stall_life............0xFF<br>
vl_enforce...............0x30<br>
m_key_violations.........0x0<br>
p_key_violations.........0x0<br>
q_key_violations.........0x0<br>
guid_cap.................0x20<br>
client_reregister........0x0<br>
mcast_pkey_trap_suppr....0x0<br>
subnet_timeout...........0x0<br>
resp_time_value..........0x10<br>
error_threshold..........0xF0<br>
max_credit_hint..........0x0<br>
link_round_trip_latency..0x0<br>
capability_mask2.........0x0<br>
link_speed_ext_active....0x0<br>
link_speed_ext_supported.0x0<br>
link_speed_ext_enabled...0x0<br>
Jun 07 14:56:58 088495 [193D0700] 0x08 -> Capability Mask:<br>
IB_PORT_CAP_HAS_TRAP<br>
IB_PORT_CAP_HAS_AUTO_MIG<br>
IB_PORT_CAP_HAS_SL_MAP<br>
IB_PORT_CAP_HAS_LED_INFO<br>
IB_PORT_CAP_HAS_SYS_IMG_GUID<br>
IB_PORT_CAP_HAS_VEND_CLS<br>
IB_PORT_CAP_HAS_CAP_NTC<br>
IB_PORT_CAP_HAS_CLIENT_REREG<br>
Jun 07 14:56:58 088499 [193D0700] 0x04 -> osm_pi_rcv_process: Discovered port num 1 with GUID 0x5ad00000c5ced for parent node GUID 0x5ad00000c5cec, TID 0x130e<br>
<br>
<br>
Then later, sm seems to have assigned a lid.<br>
<br>
Jun 07 14:56:58 090679 [161CB700] 0x08 -> PortInfo dump:<br>
port number..............1<br>
node_guid................<u></u>0x0005ad00000c5cec<br>
port_guid................<u></u>0x0005ad00000c5ced<br>
m_key....................<u></u>0x0000000000000000<br>
subnet_prefix............<u></u>0xfe80000000000001<br>
base_lid.................16<br>
master_sm_base_lid.......1<br>
capability_mask..........<u></u>0x2500A68<br>
diag_code................0x0<br>
m_key_lease_period.......0x0<br>
local_port_num...........1<br>
link_width_enabled.......0x3<br>
link_width_supported.....0x3<br>
link_width_active........0x2<br>
link_speed_supported.....0x3<br>
port_state...............INIT<br>
state_info2..............0x52<br>
m_key_protect_bits.......0x0<br>
lmc......................0x0<br>
link_speed...............0x13<br>
mtu_smsl.................0x40<br>
vl_cap_init_type.........0x30<br>
vl_high_limit............0x0<br>
vl_arb_high_cap..........0x8<br>
vl_arb_low_cap...........0x8<br>
init_rep_mtu_cap.........0x4<br>
vl_stall_life............0xFF<br>
vl_enforce...............0x30<br>
m_key_violations.........0x0<br>
p_key_violations.........0x0<br>
q_key_violations.........0x0<br>
guid_cap.................0x20<br>
client_reregister........0x1<br>
mcast_pkey_trap_suppr....0x0<br>
subnet_timeout...........0x12<br>
resp_time_value..........0x10<br>
error_threshold..........0x88<br>
max_credit_hint..........0x0<br>
link_round_trip_latency..0x0<br>
capability_mask2.........0x0<br>
link_speed_ext_active....0x0<br>
link_speed_ext_supported.0x0<br>
link_speed_ext_enabled...0x0<br>
Jun 07 14:56:58 090709 [161CB700] 0x08 -> Capability Mask:<br>
IB_PORT_CAP_HAS_TRAP<br>
IB_PORT_CAP_HAS_AUTO_MIG<br>
IB_PORT_CAP_HAS_SL_MAP<br>
IB_PORT_CAP_HAS_LED_INFO<br>
IB_PORT_CAP_HAS_SYS_IMG_GUID<br>
IB_PORT_CAP_HAS_VEND_CLS<br>
IB_PORT_CAP_HAS_CAP_NTC<br>
IB_PORT_CAP_HAS_CLIENT_REREG<br>
Jun 07 14:56:58 090713 [161CB700] 0x08 -> osm_pi_rcv_process: Client reregister received on response<br>
Jun 07 14:56:58 091294 [12FC6700] 0x10 -> osm_db_store: ]<br>
Jun 07 14:56:58 091301 [12FC6700] 0x10 -> osm_lid_mgr_process_subnet: ]<br>
Jun 07 14:56:58 091308 [161CB700] 0x10 -> pi_rcv_process_set: [<br>
Jun 07 14:56:58 091313 [161CB700] 0x08 -> pi_rcv_process_set: Received logical SetResp() for GUID 0x5ad00000c5ced, port num 1<br>
for parent node GUID 0x5ad00000c5cec TID 0x1311<br>
Jun 07 14:56:58 091320 [161CB700] 0x08 -> osm_db_update: Key:0x0005ad00000c5ced previously exists in:/var/cache/opensm/guid2mkey with value:0x0000000000000000<br>
Jun 07 14:56:58 091324 [161CB700] 0x10 -> pi_rcv_process_set: ]<br>
Jun 07 14:56:58 091327 [161CB700] 0x10 -> osm_pi_rcv_process: ]<br>
<br>
But I'm not really sure what I'm looking for.<div><div><br>
<br>
-- <br>
Orion Poplawski<br>
Technical Manager <a href="tel:303-415-9701%20x222" target="_blank" value="+13034159701">303-415-9701 x222</a><br>
NWRA, Boulder/CoRA Office FAX: <a href="tel:303-415-9702" target="_blank" value="+13034159702">303-415-9702</a><br>
3380 Mitchell Lane <a href="mailto:orion@nwra.com" target="_blank">orion@nwra.com</a><br>
Boulder, CO 80301 <a href="http://www.nwra.com" target="_blank">http://www.nwra.com</a><br>
</div></div></blockquote></div></div></div><br>
</blockquote></div><br>