<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Times New Roman","serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoPlainText>Hi Marty,<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Thank you for the detailed report and the history behind this issue. The arbitration committee understands that this is an unresolved issue that cannot be resolved by DDN and therefore we propose that we add the DDN S2A 9900 to the OFA Logo list by marking the Link Init test as Pass with Comment. The proposed language is as follows:<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><div class=MsoPlainText align=center style='text-align:center'><hr size=2 width="100%" noshade style='color:#7030A0' align=center></div><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The DDN 9900 DDR SRP target was unable to properly link with the Intel (QLogic) HCAs. A link was established, but at 4X SDR versus the expected DDR speed. This is due to a known issue with Intel HCAs and certain early Mellanox-based DDR solutions used in the DDN S2A 9900. <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText><b>Note</b>: As per past agreements, the drivers for the Intel HCAs were scheduled to have been updated to be make them compatible with the legacy Mellanox DDR Auto Negotiation specification. As this has not occurred, the inability of the DDN S2A 9900 to link at the proper speed is waived from the OFA Logo requirements; however, the link speed issue still remains. This result is being marked as a Pass with Comments to draw attention to this compatibility issue with a device that is not on the OFA Logo list.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><div class=MsoPlainText align=center style='text-align:center'><hr size=2 width="100%" noshade style='color:#7030A0' align=center></div><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The purpose of the OFA Logo testing is to assure end users that the products on the list have been thoroughly tested for interoperability issues and therefore we feel obligated to note this issue in the report.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Please let me know if this is acceptable to DDN.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Thank you,<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'>Rupert Dance<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'>Co-Chair OFA IWG<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'>Software Forge <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'>2 Greenleaf Woods drive #301<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'>Portsmouth, NH 03801<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'>Phone: 603-319-8486<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt;font-family:"Times New Roman","serif"'><a href="http://www.soft-forge.com/"><span style='color:blue'>www.soft-forge.com</span></a><o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>-----Original Message-----<br>From: Marty Schlining [mailto:mschlining@ddn.com] <br>Sent: Monday, July 16, 2012 3:05 PM<br>To: Rupert Dance<br>Subject: FW: Arbitration Request for the S2A9900 for Logo Event OFED 1.5.4.1.</p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Sorry, Rupert. Sent it the first time to an older e-mail address.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>-----Original Message-----<o:p></o:p></p><p class=MsoPlainText>From: Marty Schlining <o:p></o:p></p><p class=MsoPlainText>Sent: Monday, July 16, 2012 3:03 PM<o:p></o:p></p><p class=MsoPlainText>To: 'Rupert Dance'<o:p></o:p></p><p class=MsoPlainText>Cc: OFA Lab; Gregory Leonard; Jon Giles<o:p></o:p></p><p class=MsoPlainText>Subject: Arbitration Request for the S2A9900 for Logo Event OFED 1.5.4.1.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The S2A 9900 failed 1 test during the recent Logo event for OFED 1.5.4.1. The failure is related to link initialization. <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The S2A 9900 is a robust highly scalable storage platform. It comes in 2 versions, 8 x FC-8 ports and 8 x DDR IB ports. This system is used in a variety of applications worldwide from HPC, rich-media content development and delivery, cloud content delivery, and banking. The S2A 9900 DDR IB connected storage is in widespread use throughout the world as primary and nearline storage. Customers range from commercial to government to military. The S2A 9900 is still being sold and will require support for the next few years. It is highly likely that one of these HCA types will be connected directly to the S2A 9900 when using OFED 1.5.4.1.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The S2A9900 also received a Logo for OFED 1.5.4.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText><a href="https://www.iol.unh.edu/services/testing/ofa/interoplist/2011-12/OFA-IWG_2011-Dec_Logo-Report-DDN-Passed_SRP-Target-signed.pdf"><span style='color:windowtext;text-decoration:none'>https://www.iol.unh.edu/services/testing/ofa/interoplist/2011-12/OFA-IWG_2011-Dec_Logo-Report-DDN-Passed_SRP-Target-signed.pdf</span></a><o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The S2A9900 is deployed in the majority of the world's top 100 supercomputers.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Some other notable deployments:<o:p></o:p></p><p class=MsoPlainText>Large Data Program (U.S. Naval Research Laboratory) Xbox Live Boeing Oak Ridge National Labs Spider (World's Largest Lustre Filesystem) CEA (Tera 10 Supercomputer) NASCAR Technicolor Disney/Pixar NASA NOAA Microsoft Universal Studios<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The S2A 9900 uses "legacy" DDR HCAs. That is, DDR HCAs that were produced before the adoption of the IBTA 1.2 DDR auto-negotiation (AN) specification. The DDR HCAs in the S2A 9900 use a Mellanox proprietary (now published) auto-negotiation technique. This AN is embedded in the hardware of the S2A 9900's HCAs and cannot be changed according to the engineers at Mellanox.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Link to Mellanox's Legacy DDR AN:<o:p></o:p></p><p class=MsoPlainText><a href="http://www.iol.unh.edu/services/testing/ofa/training/Mellanox_IB_DDR_Auto-negotiation_Specification_1_0.pdf"><span style='color:windowtext;text-decoration:none'>http://www.iol.unh.edu/services/testing/ofa/training/Mellanox_IB_DDR_Auto-negotiation_Specification_1_0.pdf</span></a><o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The document is also attached to this e-mail.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>1.) The S2A 9900 links at SDR rates instead of DDR rates to the Intel (Qlogic) 7340 and Intel (Qlogic) 7342 HCAs.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Intel (Qlogic) has not never made changes to their driver to incorporate the Mellanox DDR AN specification. Also, the compat_ddr_negotiate flag of the ib_qib driver has never functioned properly with the S2A 9900. In OFED 1.5.4, the default mode did not negotiate at all with the S2A 9900. Fortunately in OFED 1.5.4.1, it at least negotiates at SDR rates using the default driver settings.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Intel (Qlogic) has made changes to their switch products (12300 and 12200 QDR IB switch) that have made those switches and the S2A 9900 fully compatible. However, no changes have ever been made to the ib_qib driver in OFED to make it fully compatible at DDR rates with the S2A 9900.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>We feel that any limitations with either the QLE7342 and/or the S2A9900 should be documented. The fact that a method exists to make the QLE7342 and the S2A 9900 link at SDR rates should put both in the "warning" category for Logo testing. It does require the use of a special flag when loading the ib_qib driver to get the link to consistently come up at SDR rates. That should also be documented. It may also be considered appropriate to use this flag when performing link initialization tests between the QLE7342 and the S2A 9900.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Ideally, we feel Intel (Qlogic) should modify their driver to fully incorporate the pre-IBTA 1.2 AN and link at DDR rates to the S2A 9900. This would be of great benefit to many who have attempted to use Intel (Qlogic) HCAs connected directly to legacy HCAs.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Regards,<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Martin Schlining<o:p></o:p></p><p class=MsoPlainText>Sr. Software Engineer<o:p></o:p></p><p class=MsoPlainText>DataDirect Networks<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Attachment saved to <<a href="file:///C:\Users\rsdance\Documents\Lamprey\OpenIB\Plugfest\2012-04\Arbitration\Mellanox_IB_DDR_Auto-negotiation_Specification_1_0.pdf"><span style='color:windowtext;text-decoration:none'>file:///C:\Users\rsdance\Documents\Lamprey\OpenIB\Plugfest\2012-04\Arbitration\Mellanox_IB_DDR_Auto-negotiation_Specification_1_0.pdf</span></a>><o:p></o:p></p></div></body></html>