<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=US-ASCII">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: Some OpenSM 1.8.0 Anomalies</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>> On Fri, 2005-09-09 at 13:22, Eitan Zahavi wrote:</FONT>
<BR><FONT SIZE=2>> > Hal Rosenstock wrote:</FONT>
<BR><FONT SIZE=2>> > > On Thu, 2005-09-08 at 09:02, Eitan Zahavi wrote:</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > >>Hal Rosenstock wrote:</FONT>
<BR><FONT SIZE=2>> > >></FONT>
<BR><FONT SIZE=2>> > >>>>>>>Sep 06 15:41:48 725691 [B76A4C40] -> __match_notice_to_inf_rec: ERR 0207:</FONT>
<BR><FONT SIZE=2>> Cannot find destination port with LID:0x0007</FONT>
<BR><FONT SIZE=2>> > >>>>>></FONT>
<BR><FONT SIZE=2>> > >>>>>>This means that the LID of the port registered as the source for this inform</FONT>
<BR><FONT SIZE=2>> info is not recognized as a valid LID.</FONT>
<BR><FONT SIZE=2>> > >>>>>></FONT>
<BR><FONT SIZE=2>> > >>>>>></FONT>
<BR><FONT SIZE=2>> > >>>>>></FONT>
<BR><FONT SIZE=2>> > >>>>>>>...</FONT>
<BR><FONT SIZE=2>> > >>>>>>>Sep 06 15:41:48 726186 [B76A4C40] -> __match_notice_to_inf_rec: ERR 0207:</FONT>
<BR><FONT SIZE=2>> Cannot find source port with GUID:0x0008f10403960559</FONT>
<BR><FONT SIZE=2>> > >>>>>></FONT>
<BR><FONT SIZE=2>> > >>>>>>The meaning of this is that the incoming trap source is not a recognized</FONT>
<BR><FONT SIZE=2>> (included in the SM database) guid</FONT>
<BR><FONT SIZE=2>> > >>>>></FONT>
<BR><FONT SIZE=2>> > >>>>></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > >>>It looks like it occurs on SM port down which seems OK.</FONT>
<BR><FONT SIZE=2>> > >></FONT>
<BR><FONT SIZE=2>> > >>OK that explains it:</FONT>
<BR><FONT SIZE=2>> > >>The errors are when the SM port has turned down. In that case all the ports that</FONT>
<BR><FONT SIZE=2>> were previously</FONT>
<BR><FONT SIZE=2>> > >>found on the fabric are now inaccessible. The SM should Report(Notice with trap</FONT>
<BR><FONT SIZE=2>> #65) for each of these ports.</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > Right, GID out of service should be and is indicated.</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > >>For that sake it scans through the InformInfo database.</FONT>
<BR><FONT SIZE=2>> > >>Apparently an InformInfo with LID=7 has requested for this report.</FONT>
<BR><FONT SIZE=2>> > >>But LID 7 does not exist anymore</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > It exists. It is just not reachable via GS (SA) LID routed packets.</FONT>
<BR><FONT SIZE=2>> > Well from the point of view of the SM it does not once the SM can not reach it.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> OK.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > >> - so the first message is valid:</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > Not sure what you mean exactly by valid here.</FONT>
<BR><FONT SIZE=2>> > Valid means that it is correct. The destination port to send the Report to is not part</FONT>
<BR><FONT SIZE=2>> of any partition any more.</FONT>
<BR><FONT SIZE=2>> > I would rephrase the error message and make it Info. There is no ERROR in loosing</FONT>
<BR><FONT SIZE=2>> some ports.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Right. This should be made into something less than error.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > >> > Sep 06 15:41:48 725691 [B76A4C40] -> __match_notice_to_inf_rec: ERR 0207:</FONT>
<BR><FONT SIZE=2>> Cannot find destination port with LID:0x0007</FONT>
<BR><FONT SIZE=2>> > >>(actually this should have caused the InformInfo record to be deleted... which I do</FONT>
<BR><FONT SIZE=2>> not think happening)</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > What should have caused the InformInfo record to be deleted ?</FONT>
<BR><FONT SIZE=2>> > "o13-17.1.2: If a Set(InformInfo) specified a valid trap source at the time of</FONT>
<BR><FONT SIZE=2>> > subscription (see o13-14.1.1: on page 746), yet Trap() forwarding fails because</FONT>
<BR><FONT SIZE=2>> > the subscriber and trap source are no longer permitted to access</FONT>
<BR><FONT SIZE=2>> > each other according to current partitioning (see o13-17.1.1: on page</FONT>
<BR><FONT SIZE=2>> > 747), then the manager shall permanently discontinue all event forwarding</FONT>
<BR><FONT SIZE=2>> > caused by the Set(InformInfo) which created a subscription to</FONT>
<BR><FONT SIZE=2>> > that trap source, except if InformInfo:LIDRangeBegin was 0xFFFF; in the</FONT>
<BR><FONT SIZE=2>> > latter case, event forwarding is discontinued only for the now-invalid trap</FONT>
<BR><FONT SIZE=2>> > source."</FONT>
<BR><FONT SIZE=2>> > Later on the same page:</FONT>
<BR><FONT SIZE=2>> > "Note also that "permanently discontinue all event forwarding" is meant to</FONT>
<BR><FONT SIZE=2>> > indicate that the subscription for forwarding is dropped by the manager; if</FONT>
<BR><FONT SIZE=2>> > the source later becomes reachable again by the subscriber, a new</FONT>
<BR><FONT SIZE=2>> > Set(InformInfo) is required to re-establish event forwarding, if that is what</FONT>
<BR><FONT SIZE=2>> > is desired. (This may not be desired; when the source becomes reachable</FONT>
<BR><FONT SIZE=2>> > again, it may have acquired new characteristics, such as new, different</FONT>
<BR><FONT SIZE=2>> > software functions, that make such forwarding inappropriate.)"</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > > This error being detected ?</FONT>
<BR><FONT SIZE=2>> > Not currently</FONT>
<BR><FONT SIZE=2>> > > If so, should it wait for the error or should it occur</FONT>
<BR><FONT SIZE=2>> > > when the SM port goes down do this (clear the inform list perhaps with</FONT>
<BR><FONT SIZE=2>> > > the exception of the local node) ?</FONT>
<BR><FONT SIZE=2>> > Maybe or just code the generic code to handle 013-17.1.2</FONT>
<BR><FONT SIZE=2>> > > That would require/mean</FONT>
<BR><FONT SIZE=2>> > > reregistration is required when the node comes back. SA clients won't</FONT>
<BR><FONT SIZE=2>> > > necessarily do this when the SM port comes back without something like</FONT>
<BR><FONT SIZE=2>> > > ClientReregistration.</FONT>
<BR><FONT SIZE=2>> > Correct. This is another reason why ClientReRegistration is an important feature of</FONT>
<BR><FONT SIZE=2>> the</FONT>
<BR><FONT SIZE=2>> > access layer.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I would have ended that sentence after feature. It does not need to be</FONT>
<BR><FONT SIZE=2>> implemented in the access layer.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > >>Later we see the following error:</FONT>
<BR><FONT SIZE=2>> > >> > Sep 06 15:41:48 726186 [B76A4C40] -> __match_notice_to_inf_rec: ERR 0207:</FONT>
<BR><FONT SIZE=2>> Cannot find source port with GUID:0x0008f10403960559</FONT>
<BR><FONT SIZE=2>> > >>This is sent during the section where node 0x0008f10403960559 is being teared off</FONT>
<BR><FONT SIZE=2>> from the SMDB.</FONT>
<BR><FONT SIZE=2>> > >></FONT>
<BR><FONT SIZE=2>> > >>The code in osm_inform.c say:</FONT>
<BR><FONT SIZE=2>> > >> /* Check if there is a pkey match. o13-17.1.1*/</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > Where is this performed ?</FONT>
<BR><FONT SIZE=2>> > osm_inform.c</FONT>
<BR><FONT SIZE=2>> > __match_notice_to_inf_rec</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > >> /* Check if the issuer of the trap is the SM. If it is, then the pkey</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > ^^</FONT>
<BR><FONT SIZE=2>> > > gid</FONT>
<BR><FONT SIZE=2>> > The requirement is to have a shared PKey according to PKey sharing rules between</FONT>
<BR><FONT SIZE=2>> the</FONT>
<BR><FONT SIZE=2>> > InformInfo requester and the Trap generator. However, in the case of traps 64-67</FONT>
<BR><FONT SIZE=2>> > the SM is the Trap generator. So we need the spacial logic below to obtain the port</FONT>
<BR><FONT SIZE=2>> gid</FONT>
<BR><FONT SIZE=2>> > that the trap refers to from within the notice data details fields and not from the</FONT>
<BR><FONT SIZE=2>> issuer field.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I think the comment in the code is wrong here and should be gid rather</FONT>
<BR><FONT SIZE=2>> than pkey. I do agree that the pkey sharing needs checking but that is</FONT>
<BR><FONT SIZE=2>> separate.</FONT>
<BR><FONT SIZE=2>[EZ] OK we can improve the comments accuracy and readability. As always comments written by the developers are somewhat biased by the fact he already understands the code. So the first time reader can do a better job...</FONT></P>
<P><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> > >> comparison should be done on the trap source (saved as the gid in the</FONT>
<BR><FONT SIZE=2>> > >> data details field).</FONT>
<BR><FONT SIZE=2>> > >> If the issuer gid is not the SM - then it is the guid of the trap</FONT>
<BR><FONT SIZE=2>> > >> source. */</FONT>
<BR><FONT SIZE=2>> > >> if ( (cl_ntoh64(p_ntc->issuer_gid.unicast.prefix) == p_subn->opt.subnet_prefix)</FONT>
<BR><FONT SIZE=2>> &&</FONT>
<BR><FONT SIZE=2>> > >> (cl_ntoh64(p_ntc->issuer_gid.unicast.interface_id) == p_subn->sm_port_guid)</FONT>
<BR><FONT SIZE=2>> )</FONT>
<BR><FONT SIZE=2>> > >> {</FONT>
<BR><FONT SIZE=2>> > >> /* The issuer is the SM this is trap 64-67 - compare the pkey</FONT>
<BR><FONT SIZE=2>> > >> with the gid saved on the data details. */</FONT>
<BR><FONT SIZE=2>> > >> source_gid = p_ntc->data_details.ntc_64_67.gid;</FONT>
<BR><FONT SIZE=2>> > >> }</FONT>
<BR><FONT SIZE=2>> > >> else</FONT>
<BR><FONT SIZE=2>> > >> {</FONT>
<BR><FONT SIZE=2>> > >> source_gid = p_ntc->issuer_gid;</FONT>
<BR><FONT SIZE=2>> > >> }</FONT>
<BR><FONT SIZE=2>> > >></FONT>
<BR><FONT SIZE=2>> > >>In our case the trap is 65 and sent by the SM. However, the spec required to check</FONT>
<BR><FONT SIZE=2>> > >>the tear down port and the target of the Report will share a PKey.</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > I'm not sure what you are referring to in the spec. In any case,</FONT>
<BR><FONT SIZE=2>> > > shouldn't the local ports perhaps be an exception to this ?</FONT>
<BR><FONT SIZE=2>> > I do not think so. The requirement make sense for all traps:</FONT>
<BR><FONT SIZE=2>> > If the Trap describes a port A then it should not be forwarded to another port B</FONT>
<BR><FONT SIZE=2>> unless they</FONT>
<BR><FONT SIZE=2>> > share a PKey:</FONT>
<BR><FONT SIZE=2>> > "o13-17.1.1: Managers that support event forwarding and have confirmed</FONT>
<BR><FONT SIZE=2>> > a request for event subscription shall forward corresponding events to the</FONT>
<BR><FONT SIZE=2>> > subscriber using a Report(Notice) MAD, as long as the subscriber and</FONT>
<BR><FONT SIZE=2>> > Trap() source are permitted to access each other according to current partitioning."</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > >> In out case the</FONT>
<BR><FONT SIZE=2>> > >>source of the event is considered to be the port that is tear down. (As we want to</FONT>
<BR><FONT SIZE=2>> > >>prevent any case where port not sharing PKey will get reports on each other).</FONT>
<BR><FONT SIZE=2>> > >>But since the "source" port is being teared down we can not find it's PKey table ...</FONT>
<BR><FONT SIZE=2>> > >>(actually we look first in the Port by LID table - and can not find it).</FONT>
<BR><FONT SIZE=2>> > >></FONT>
<BR><FONT SIZE=2>> > >>This means we will never send Report(Notice trap#65) to any node.</FONT>
<BR><FONT SIZE=2>> > >>How do we solve that bug? Maybe we have a way to find the "source" port PKey</FONT>
<BR><FONT SIZE=2>> that</FONT>
<BR><FONT SIZE=2>> > >>is not yet corrupted.</FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > ></FONT>
<BR><FONT SIZE=2>> > > I'm not totally following this because of the PKey v. GID issue above and</FONT>
<BR><FONT SIZE=2>> > > I think local ports may be (needed to be) treated differently.</FONT>
<BR><FONT SIZE=2>> > I hope the above 17.1.1 convinced you. The GID vs PKey is just unclear</FONT>
<BR><FONT SIZE=2>> documentation.</FONT>
<BR><FONT SIZE=2>> > The idea is that for trap# 64-67 which are generated by the SM you can not simply</FONT>
<BR><FONT SIZE=2>> use the SM PKey but</FONT>
<BR><FONT SIZE=2>> > lookup the gid of the reported port from within the notice data details and then</FONT>
<BR><FONT SIZE=2>> lookup that port PKey.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> OK. I'm convinced.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> I'm still not sure what is the bug you are referring to above though.</FONT>
<BR><FONT SIZE=2>[EZ] The bug is that the code does not perform the required operations to meet o13-17.1.2 compliancy.</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> -- Hal</FONT>
</P>
</BODY>
</HTML>