<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Cambria;
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;}
@font-face
{font-family:STXihei;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle24
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle25
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle26
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle27
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle28
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle29
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle30
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle31
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle32
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle33
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle34
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle35
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle36
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle37
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle38
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:331494753;
mso-list-type:hybrid;
mso-list-template-ids:-2131211314 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1
{mso-list-id:556473629;
mso-list-type:hybrid;
mso-list-template-ids:-563845728 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:1857302802;
mso-list-type:hybrid;
mso-list-template-ids:-1845313730 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l2:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1027" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Yong,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> I have been checking it out. No issues so far. In your minor changes, please put Dumpbuffersize #define in uppercase.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"> -Rick<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> nvmewin-bounces@lists.openfabrics.org [mailto:nvmewin-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>Yong Chen<br>
<b>Sent:</b> Wednesday, January 08, 2014 7:54 PM<br>
<b>To:</b> Alex Chang; Foster, Carolyn D<br>
<b>Cc:</b> nvmewin@lists.openfabrics.org<br>
<b>Subject:</b> Re: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, Alex and all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Where are we now on this review? Alex do you need another explicit approval from gatekeepers?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:nvmewin-bounces@lists.openfabrics.org">nvmewin-bounces@lists.openfabrics.org</a> [<a href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>Yong Chen<br>
<b>Sent:</b> Thursday, December 19, 2013 4:26 PM<br>
<b>To:</b> Alex Chang; Foster, Carolyn D<br>
<b>Cc:</b> <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I finished x86 tests on Win 7 & 8. I hope we are wrapping up the review soon? The revision to previous changes is very minor.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Alex, <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">What is your release schedule for 1.3? I will be mostly oof during the next 2 weeks.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Alex Chang [<a href="mailto:Alex.Chang@pmcs.com">mailto:Alex.Chang@pmcs.com</a>]
<br>
<b>Sent:</b> Monday, December 16, 2013 6:33 PM<br>
<b>To:</b> Yong Chen; Foster, Carolyn D<br>
<b>Cc:</b> <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I am forwarding the patch Yong sent out today. Please find them in the attachment. Should you have any questions/feedbacks, please rely this message. Please review the changes and test it with your devices as
well.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Yong Chen [<a href="mailto:Yong.sc.Chen@huawei.com">mailto:Yong.sc.Chen@huawei.com</a>]
<br>
<b>Sent:</b> Monday, December 16, 2013 3:36 PM<br>
<b>To:</b> Foster, Carolyn D; Alex Chang<br>
<b>Cc:</b> <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, Alex and all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I have uploaded the changes to the web and Alex will reply back with zipped source code when received it. Thank you, Alex!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The recursive callstack is now fixed, which was a serious offense. Thanks Carolyn, for having brought it up. I also have addressed all points raised by Alex or Rick.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The other major topic is memory usage in dump mode: I have reserved 5x 64KB, the actually usage is about 2.2x 64KB. The bulk of them is used by LunExt (0x11000=68KB), almost half. The variable size of all IO
queues are very tiny, along other normal usage. 4x 64KB is probably enough, but to double this 2.2 to 5 is more prudent choice, IMO and that is the value that has been tested all along.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I have finished with SDStress and ioMeter tests (including win7 x86) and pwrtest. Two more tests remain:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">A: installing Win8 now; <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">B: Due to the destructive-nature of SCSICompliance test, I will leave that to the last because I have only 1 controller to test with. I don’t expect any changes from the initialization refactoring would affect
SCSI so I am sending out now so you guys can review while I am finishing up the remaining tests.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Please let me or Alex know if you need the binaries as well.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:nvmewin-bounces@lists.openfabrics.org">nvmewin-bounces@lists.openfabrics.org</a> [<a href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>Yong Chen<br>
<b>Sent:</b> Tuesday, November 26, 2013 2:13 PM<br>
<b>To:</b> Foster, Carolyn D; Judy Brock-SSI; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">See inline.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Foster, Carolyn D [<a href="mailto:carolyn.d.foster@intel.com">mailto:carolyn.d.foster@intel.com</a>]
<br>
<b>Sent:</b> Tuesday, November 26, 2013 12:09 PM<br>
<b>To:</b> Yong Chen; Judy Brock-SSI; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">See my comments below in green. Also I found two more issues:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">According to MSDN StorportEnablePassiveInitialization will fail if the system does not support DPCs, which is the case in crashdump mode. We should have a check for ntldrDump in NVMeInitialize and call NVMePassiveInitialize
directly if it’s true.<o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="color:#1F497D">[Yong>] are you talking about this block? You are right, that is exactly the case.<br>
</span></i></b><span style="font-size:8.0pt;color:#1F497D"> /*<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> * When Crashdump/Hibernation driver is being loaded, need to complete the<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> * entire initialization here. In the case of normal driver loading, enable<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> * passive initialization and let NVMePassiveInitialization handle the rest<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> * of the initialization<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> */<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> if (pAE->ntldrDump == FALSE) {<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:8.0pt;color:#1F497D">…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> /* Call StorPortPassiveInitialization to enable passive init */<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"> StorPortEnablePassiveInitialization(pAE, NVMePassiveInitialize);<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The other potential issue is that in NVMeCallArbiter, in ntldrDump mode we will call NVMeRunning. This could cause a very deep call stack since NVMeCallArbiter is also called from NVMeRunning. In hibernate
mode we have limited memory and this could cause issues. I suggest making modifications to NVMeRunningStartAttempt around the NVMeRunning call. It could be a while-loop that would call NVMeRunning if ntldrDump is true, with a stall execution, that would
loop until the nextDriverState is start complete, or failed. <o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="color:#1F497D">[Yong>] you are right about the call stack. In the while loop we could create a new separate mini state-machine for the dump mode initialization. Given more time I can experiment with it.</span></i></b><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Carolyn<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Yong Chen [<a href="mailto:Yong.sc.Chen@huawei.com">mailto:Yong.sc.Chen@huawei.com</a>]
<br>
<b>Sent:</b> Monday, November 25, 2013 11:23 PM<br>
<b>To:</b> Foster, Carolyn D; Judy Brock-SSI; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, Carolyn,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks for the input.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">There is no DPC or interrupt in dump mode. It is the polling method to call the same routine. Could you elaborate why you don’t expect DPCs to work properly on Win7?
</span><span style="color:#00B050">I couldn’t find where polling mode is set in the hibernate path, and I was also expecting to see the DPC initialization calls wrapped in ntldrDump checks. Specifically in NVMePassiveInitialize.
</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><b><i><span style="color:#1F497D">[Yong>] in dump mode the driver model behaves as if in polling mode. Not something explicit to set. </span></i></b><span style="color:#00B050">NVMePassiveInitialize (&DPC initialization
calls)</span><b><i><span style="color:#1F497D"> won’t be issued in dump mode. see first comment.</span></i></b><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="color:#1F497D"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">The small buffer is reserved during normal driver load. In dump mode you can’t allocate anything (doc says paltry 64KB). That dump buffer is guaranteed when entering dump mode. Regardless, the same
code path would fail not BSOD if BUFFER is NULL as in any other cases. </span><span style="color:#00B050">I’m specifically asking about line 189 in nvmeInit. It seems possible for that line of code, the StorportAllocateContiguousMemory to be called in the
crashdump path. Can you confirm that function call in crashdump/hibernate mode simply fails? If it does then I agree that a null buffer here will likely not crash the system.</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="color:#1F497D">[Yong>] I see. This new API call is recently merged change. I have not got into this failure case. From my earlier experience with other allocation function (not this new API), all these APIs simply fail
with NULL returned.</span></i></b><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="color:#1F497D"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">I added the code because for a long time I was and still am dealing with engineer-sample hardware not finished products. After several revisions, they are much more mature now than earlier this year.
This is how I make them in consistent ready mode.<br>
The rationale for cycling of CC.EN bit during resuming from hibernation is just to mirror normal Initialization step.<br>
The timeout is predefined value STORPORT_TIMER_CB_us.<br>
For mal-function hardware, the same logic would already have experienced same problems in NVMeInitialize() at raised level.<br>
The best way to decide is to test on different implementation of NVMe devices from various vendors and see whether we need to tune these values.
</span><span style="color:#00B050"> My concern here is that the RecoveryDpc routine is not just called during hibernate, it is called during runtime if windows needs to reset the controller. I’m concerned with how these changes impact the normal runtime driver.
Did you test this function during runtime? What happens if the maximum time is spent in it?</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><i><span style="color:#1F497D">[Yong>] yes, it will be called by windows if hardware misbehaves, trying to reset the hardware. I didn’t try to simulate this scenario, not easy with real hardware. With this change, this same reset is
being exercised every time in resuming from hibernation. we can find out by turning on fault injections, however it is not something we have been running regularly.<o:p></o:p></span></i></b></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Hope these help, <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Foster, Carolyn D [<a href="mailto:carolyn.d.foster@intel.com">mailto:carolyn.d.foster@intel.com</a>]
<br>
<b>Sent:</b> Monday, November 25, 2013 3:25 PM<br>
<b>To:</b> Judy Brock-SSI; Yong Chen; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi Yong, I also have some feedback and questions about the code changes.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in">1.<span style="font-size:7.0pt;font-family:"Times New Roman","serif""> </span>I am surprised that there are no ntldrDump checks around the DPC initialization and calls. I wouldn’t have expected the
DPCs to work properly on the Windows 7 systems. <o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in">2.<span style="font-size:7.0pt;font-family:"Times New Roman","serif"">
</span>In function NVMeAllocateMem, the ntldrDump check is wrapped such that if no buffer is allocated from the DumpBuffer, the code path could end up calling StorPortAllocateContiguousMemory in ntldrDump mode. Will this cause a BSOD? Or will it just fail?<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in">3.<span style="font-size:7.0pt;font-family:"Times New Roman","serif"">
</span>In RecoveryDpcRoutine new code has been added above the reset adapter call not related to ntldrDump. If the controller isn’t responding, this additional delay time could cause a DPC watchdog timeout bugcheck if the maximum time allowed for a DPC is
exceeded. I have some concerns about this new code, what was your reasoning for adding it? <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Carolyn<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:nvmewin-bounces@lists.openfabrics.org">nvmewin-bounces@lists.openfabrics.org</a> [<a href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>Judy Brock-SSI<br>
<b>Sent:</b> Monday, November 25, 2013 3:18 PM<br>
<b>To:</b> Yong Chen; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi Yong,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Ensuring CSTS.RDY == 0 before setting CC.EN to ‘1’ is required by the spec – it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is
a sanity check that needs to take place <u>immediately </u>before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I don’t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you
are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn’t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don’t enhance
your current changes one way or the other.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I say, put the fix in – it’s fairly trivial.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Judy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Yong Chen [<a href="mailto:Yong.sc.Chen@huawei.com">mailto:Yong.sc.Chen@huawei.com</a>]
<br>
<b>Sent:</b> Monday, November 25, 2013 10:53 AM<br>
<b>To:</b> Judy Brock-SSI; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, Judy,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine()
and NVMeInitialize(). <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change
the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">What do other folks think?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Judy Brock-SSI [<a href="mailto:judy.brock@ssi.samsung.com">mailto:judy.brock@ssi.samsung.com</a>]
<br>
<b>Sent:</b> Sunday, November 24, 2013 8:20 AM<br>
<b>To:</b> Yong Chen; Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">
nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">Hi Yong,<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ‘1’.<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself.
If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn’t check before calling.<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure:<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green;background:yellow;mso-highlight:yellow"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green;background:yellow;mso-highlight:yellow"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green;background:yellow;mso-highlight:yellow"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green;background:yellow;mso-highlight:yellow">* @return BOOLEAN<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green;background:yellow;mso-highlight:yellow">* TRUE - If Adapter is enabled correctly<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green;background:yellow;mso-highlight:yellow">* FALSE - If anything goes wrong</span><span style="font-family:"Courier New";color:green"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green">******************************************************************************/<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">BOOLEAN</span><span style="font-family:"Courier New""> NVMeEnableAdapter(<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> PNVME_DEVICE_EXTENSION pAE<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">)<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">{<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> PQUEUE_INFO pQI = &pAE->QueueInfo;<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> NVMe_CONTROLLER_CONFIGURATION CC = {0};<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">
<span style="background:yellow;mso-highlight:yellow">NVMe_CONTROLLER_STATUS CSTS = {0};</span><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">
<span style="background:yellow;mso-highlight:yellow">ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us;<o:p></o:p></span></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> ULONG PollCount;</span><span style="font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New"">
<span style="color:green">/*<o:p></o:p></span></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green"> * Program Admin queue registers before enabling the adapter:<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green"> * Admin Queue Attributes<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New";color:green"> */<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> StorPortWriteRegisterUlong(<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> pAE,<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> (PULONG)(&pAE->pCtrlRegister->AQA),<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> (pQI->pSubQueueInfo->SubQEntries - 1) +<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-family:"Courier New""> ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB));<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt">.<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt">.<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:.5in"><b><span style="font-size:12.0pt">.<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin-left:.5in"><i><span style="font-family:"Courier New"">(further down):<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New"">StorPortDebugPrint(INFO,
<span style="color:#A31515">"NVMeEnableAdapter: Setting EN...\n"</span>);<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New"">
<span style="color:green">/*<o:p></o:p></span></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";color:green"> * Set up Controller Configuration Register<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";color:green"> */<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New"">
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New"">
<span style="color:green;background:yellow;mso-highlight:yellow">/* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */<o:p></o:p></span></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> <span style="color:blue">for</span> (PollCount = 0; PollCount < PollMax; PollCount++) {<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> CSTS.AsUlong = StorPortReadRegisterUlong(pAE,<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong));<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<span style="color:blue">if</span> (CSTS.RDY == 0) {<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<span style="color:green">/* Move on if RDY bit is cleared */<o:p></o:p></span></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<span style="color:blue">break</span>;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> }<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> NVMeStallExecution(pAE, MAX_STATE_STALL_us);<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> }<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<span style="color:blue">if</span> (CSTS.RDY != 0) {<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<span style="color:green">/* If RDY bit won't clear we can't enable the adapter */<o:p></o:p></span></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow">
<span style="color:blue">return</span> FALSE;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New";background:yellow;mso-highlight:yellow"> }</span><span style="font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New"">
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.EN = 1;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.CSS = NVME_CC_NVM_CMD;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT);<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.AMS = NVME_CC_ROUND_ROBIN;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.SHN = NVME_CC_SHUTDOWN_NONE;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.IOSQES = NVME_CC_IOSQES;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.IOCQES = NVME_CC_IOCQES;<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> StorPortWriteRegisterUlong(pAE,<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> (PULONG)(&pAE->pCtrlRegister->CC),<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New""> CC.AsUlong);<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-autospace:none"><span style="font-family:"Courier New"">
<span style="color:blue;background:yellow;mso-highlight:yellow">return</span><span style="background:yellow;mso-highlight:yellow"> TRUE;</span><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-family:"Courier New"">}
<span style="color:green">/* NVMeEnableAdapter */</span></span><b><span style="font-size:16.0pt"><o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Judy<o:p></o:p></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:nvmewin-bounces@lists.openfabrics.org">nvmewin-bounces@lists.openfabrics.org</a> [<a href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>Yong Chen<br>
<b>Sent:</b> Friday, November 22, 2013 1:48 PM<br>
<b>To:</b> Alex Chang; <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, everyone,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I hope many are busy testing the changes on your devices. If you have any feedback to share, I’d very appreciate it. Holiday is upon us and we’d like to wrap up this much delayed soon.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Yong Chen
<br>
<b>Sent:</b> Wednesday, November 20, 2013 12:52 AM<br>
<b>To:</b> 'Alex Chang'; Uma Parepalli<br>
<b>Cc:</b> 'nvmewin@lists.openfabrics.org'<br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Object #2: hibernate and then resume on all client SKUs.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">+ Minor cleaning up and fixes along the way.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">High-level Summary:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver
will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code
path for IOs in dump mode.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Detailed Briefs:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo4"><![if !supportLists]><span style="font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">nvmeInit.c<br>
changes to manage buffers and IO queues in dump modes.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo4"><![if !supportLists]><span style="font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">nvmeIo.c<br>
minor tweak in dump mode where only boot-CPU is available<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo4"><![if !supportLists]><span style="font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">nvmeSnti.c<br>
fix existing bug where FLUSH cmd should include NSID (all NSs in this case).<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo4"><![if !supportLists]><span style="font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">nvmeStat.c<br>
helper function change due to some timer related API not available in dump mode.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo4"><![if !supportLists]><span style="font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:#1F497D">nvmeStd.c<br>
A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1.<br>
B: introduced same waiting logic after clearing CC.EN = 0.<br>
C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced.
<br>
<br>
Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not
by this change.<br>
(</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif"">Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.</span><span style="color:#1F497D">)
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Alex Chang [<a href="mailto:Alex.Chang@pmcs.com">mailto:Alex.Chang@pmcs.com</a>]
<br>
<b>Sent:</b> Tuesday, November 19, 2013 5:04 PM<br>
<b>To:</b> Uma Parepalli; Yong Chen<br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi Yong,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<br>
Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Uma Parepalli [<a href="mailto:uma.parepalli@skhms.com">mailto:uma.parepalli@skhms.com</a>]
<br>
<b>Sent:</b> Tuesday, November 19, 2013 4:05 PM<br>
<b>To:</b> Alex Chang<br>
<b>Subject:</b> RE: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Is there a change log file or something that explains what changes are made without looking at the code?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Uma<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:nvmewin-bounces@lists.openfabrics.org">nvmewin-bounces@lists.openfabrics.org</a> [<a href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>Alex Chang<br>
<b>Sent:</b> Tuesday, November 19, 2013 4:05 PM<br>
<b>To:</b> <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a><br>
<b>Subject:</b> Re: [nvmewin] code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<br>
Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Yong Chen [<a href="mailto:Yong.sc.Chen@huawei.com">mailto:Yong.sc.Chen@huawei.com</a>]
<br>
<b>Sent:</b> Tuesday, November 19, 2013 3:59 PM<br>
<b>To:</b> <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a>; Alex Chang<br>
<b>Subject:</b> RE: code review: crash dump & hibernation support<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:8.0pt;color:#1F497D"><Resending to due to attachment issue><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi, all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Please download the source code from the link in the attached email (you need to have Silverlight installed).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Yong<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Yong Chen
<br>
<b>Sent:</b> Monday, November 18, 2013 4:13 PM<br>
<b>To:</b> 'nvmewin@lists.openfabrics.org'; 'Alex Chang'</span><o:p></o:p></p>
<p class="MsoNormal">Hi, Alex and all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same
<span style="font-size:10.0pt;font-family:"Arial","sans-serif"">CSTS.RDY</span> issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Yong<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Tests that I have gone thru:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>manual crash dump.<br>
KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f”<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>SCSICompliance (log attached).<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">4.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>stresses: iostress, sdstress (log attached)<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">5.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>hibernation has been tested on win8.0 as well, but not extensively.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">6.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l1 level1 lfo6"><![if !supportLists]><span style="mso-list:Ignore">7.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:STXihei;color:black">Yong Chen<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:STXihei;color:black">Storage Architect<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="ZH-CN" style="font-size:10.0pt;font-family:SimSun;color:black">华为技术有限公司</span><span style="font-size:10.0pt;font-family:STXihei;color:black"> Huawei Technologies Co., Ltd<o:p></o:p></span></p>
<p class="MsoNormal"><!--[if gte vml 1]><v:shapetype id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t" path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f">
<v:stroke joinstyle="miter" />
<v:formulas>
<v:f eqn="if lineDrawn pixelLineWidth 0" />
<v:f eqn="sum @0 1 0" />
<v:f eqn="sum 0 0 @1" />
<v:f eqn="prod @2 1 2" />
<v:f eqn="prod @3 21600 pixelWidth" />
<v:f eqn="prod @3 21600 pixelHeight" />
<v:f eqn="sum @0 0 1" />
<v:f eqn="prod @6 1 2" />
<v:f eqn="prod @7 21600 pixelWidth" />
<v:f eqn="sum @8 21600 0" />
<v:f eqn="prod @7 21600 pixelHeight" />
<v:f eqn="sum @10 21600 0" />
</v:formulas>
<v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect" />
<o:lock v:ext="edit" aspectratio="t" />
</v:shapetype><v:shape id="ridImg" o:spid="_x0000_s1026" type="#_x0000_t75" alt="Company_logo" style='position:absolute;margin-left:0;margin-top:0;width:76.5pt;height:24pt;z-index:251657728;visibility:visible;mso-wrap-style:square;mso-wrap-distance-left:0;mso-wrap-distance-top:0;mso-wrap-distance-right:0;mso-wrap-distance-bottom:0;mso-position-horizontal:left;mso-position-horizontal-relative:text;mso-position-vertical:absolute;mso-position-vertical-relative:line' o:allowoverlap="f">
<v:imagedata src="cid:image001.jpg@01CF0D2B.EC7CA420" o:title="company_logo" />
<w:wrap type="square"/>
</v:shape><![endif]--><![if !vml]><img width="102" height="32" src="cid:image001.jpg@01CF0D2B.EC7CA420" align="left" alt="Company_logo" v:shapes="ridImg"><![endif]><span style="font-size:10.0pt;font-family:STXihei;color:black">Office:
408-330-5482<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Cambria","serif";color:black">Mobile: 425-922-0658</span><span style="font-size:10.0pt;font-family:STXihei;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Email: <a href="mailto:yong.sc.chen@huawei.com">
yong.sc.chen@huawei.com</a><br>
2330 Central Expressway<br>
Santa Clara, CA 95050<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><a href="http://www.huawei.com">http://www.huawei.com</a><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="ZH-CN" style="font-size:7.5pt;font-family:SimSun;color:gray">本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁</span><span style="font-size:7.5pt;font-family:STXihei;color:gray"><br>
</span><span lang="ZH-CN" style="font-size:7.5pt;font-family:SimSun;color:gray">止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中</span><span style="font-size:7.5pt;font-family:STXihei;color:gray"><br>
</span><span lang="ZH-CN" style="font-size:7.5pt;font-family:SimSun;color:gray">的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！</span><span style="font-size:7.5pt;font-family:STXihei;color:gray"><br>
</span><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:gray">This e-mail and its attachments contain confidential information from HUAWEI, which
<br>
is intended only for the person or entity whose address is listed above. Any use of the
<br>
information contained herein in any way (including, but not limited to, total or partial
<br>
disclosure, reproduction, or dissemination) by persons other than the intended <br>
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by
<br>
phone or email immediately and delete it!</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>