<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=US-ASCII">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RMPP Message Format Errors</TITLE>
</HEAD>
<BODY>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">Hi Sean, Hal,</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">We have started testing RMPP packets with osmtest and opensm</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> (gen2 version).</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">We did not go very far. The first NodeRecord GetTable of all</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">the</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> nodes in a</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">"</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">loopback</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">"</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> case, has some issues</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">.</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">The explanation is below:</FONT></P>
<P><FONT COLOR="#0000FF" FACE="Palatino Linotype">1. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">NodeRecord MAD size is 112bytes (note the required padding of 4 bytes at the end of the NodeRec data).</FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">2. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">OpenSM log file shows the query</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">should</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"></FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">return 2 records</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> one for each end-port. This really happens</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">:</FONT>
<UL><UL>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">Aug 21 14:59:49 998104 [40D9DBB0] -> __osm_nr_rcv_create_nr: Looking for NodeRecord with LID: 0x0 GUID:0x0000000000000000</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">Aug 21 14:59:49 998224 [40D9DBB0] -> __osm_nr_rcv_new_nr: New NodeRecord: node 0x0002c902000017a0</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype"> port 0x0002c902000017a1, lid 0x1.</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">Aug 21 14:59:49 998327 [40D9DBB0] -> __osm_nr_rcv_new_nr: New NodeRecord: node 0x0002c902000017a0</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype"> port 0x0002c902000017a2, lid 0x2.</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">Aug 21 14:59:49 998395 [40D9DBB0] -></FONT> <FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">osm_nr_rcv_process: Returning 2 records</FONT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">.</FONT></P>
</UL></UL>
<P><FONT COLOR="#0000FF" FACE="Palatino Linotype">3. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">On the wire</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">we see the following (see attached gif for more details):</FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">a. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">Two data segments were sent and two ACKs were returned.</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">This is OK.</FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">b. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">T</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">he</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> first segment reports PayLen = 440bytes. According to the spec the first segment might provide</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">pay</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">len</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> != 0 and when it is done it should be equal to the (class header * Num-Segments) + data length. In our case we have data length = 2*112</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">, and SA extra header = 20byte * 2seg. This leads to peylen=264 and not 440!!!<BR>
The spec defines</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">that in p775</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">-</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">l</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">37.</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"><BR>
</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">So t</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">his is a violation of the spec</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">.</FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">c. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">The last segment (segment 2) provides the paylen field of 100.</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">The</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">expected value for the last segment length should have been: SA</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">extra h</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">eader + leftover</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">data</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">size from</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">prev segments. Since</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">the</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> first segment has 200bytes for</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">data the left over should have been 112*2</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">-</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> 200 = 24. With</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">the</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> SA</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">extra</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">header</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"></FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">44bytes.<BR>
</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">So t</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">his is a</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">nother</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype"> violation of the spec</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">.</FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">d. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">The</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">analyzer is confused by the above and reports</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> the result as having 3 NodeRecords.</FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">e. </FONT><FONT FACE="Arial" SIZE=2 COLOR="#000000"> <<Gen2 NodeRec GetTable RMPP Format Error.GIF>> </FONT>
<BR><FONT COLOR="#0000FF" FACE="Palatino Linotype">4. </FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">Following that when we trace the log file of osmtest we find more issues.</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">Probably caused by changes to the vendor layer or the rmpp assembly: It is expected that after</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">assembly</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"></FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">the</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"> size of the RMPP mad reported to the</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">osm vendor layer will be the</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">rmpp header +</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">SA extra header + data-size.</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">I</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype">n our case that is</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">32 + 20 + 2*112 =</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">276.</FONT> </P>
<UL>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">The log file shows:</FONT></P>
<UL>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">Aug 21 14:59:49 [40D87BB0] -> __osmv_sa_mad_rcv_cb: Count = 1 = 200 / 112 (88)</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=1 FACE="Palatino Linotype">Aug 21 14:59:49 [4017F6C0] -> osmtest_write_all_node_recs: Received 1 records</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#FF0000" FACE="Palatino Linotype">So t</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">his is a</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype">nother</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype"></FONT> <FONT COLOR="#FF0000" FACE="Palatino Linotype">problem</FONT> <FONT COLOR="#FF0000" FACE="Palatino Linotype">-</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype"></FONT> <FONT COLOR="#FF0000" FACE="Palatino Linotype">probably</FONT><FONT COLOR="#FF0000" FACE="Palatino Linotype"> with the way RMPP results are assembled or pass back to</FONT> <FONT COLOR="#FF0000" FACE="Palatino Linotype">the</FONT> <FONT COLOR="#FF0000" FACE="Palatino Linotype">vendor.</FONT></P>
</UL></UL>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">Please let me know if you will have time to dig into these problems or if I should try</FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">and</FONT><FONT COLOR="#0000FF" FACE="Palatino Linotype"></FONT> <FONT COLOR="#0000FF" FACE="Palatino Linotype">resolve them myself and provide patches.</FONT> </P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">Thanks</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" FACE="Palatino Linotype">Eitan</FONT></P>
<P ALIGN=LEFT><B><I></I></B><A NAME="_MailAutoSig"><B><I><FONT COLOR="#0000FF" SIZE=6 FACE="Monotype Corsiva">Eitan Zahavi</FONT></I></B></A></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=2 FACE="Tahoma">Design Technology Director</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=2 FACE="Tahoma">Mellanox Technologies LTD</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=2 FACE="Tahoma">Tel:+972-4-9097208<BR>
Fax:+972-4-9593245</FONT></P>
<P ALIGN=LEFT><FONT COLOR="#0000FF" SIZE=2 FACE="Tahoma">P.O. Box 586 Yokneam 20692 ISRAEL</FONT></P>
<P ALIGN=LEFT></P>
</BODY>
</HTML>