<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:42095566;
mso-list-type:hybrid;
mso-list-template-ids:-1516065082 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1
{mso-list-id:1776709531;
mso-list-type:hybrid;
mso-list-template-ids:1685337582 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:2110468913;
mso-list-type:hybrid;
mso-list-template-ids:1166843862 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l2:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black">In our testing, we create a situation where we put the NVMe driver under heavy I/O load with Iometer and then cause the device to stop responding. This results in I/O request timeouts
which eventually causes the driver to be called at it’s HwStorResetBus entry point (NVMeResetBus). I have some feedback on the current architecture of that routine:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="color:black"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:black">Among other things, NMeResetBus schedules a DPC to complete any pending commands. This creates a situation where upon return from this entry point, there are still cmds outstanding which don’t get completed
till the DPC runs. According to the WDK, this doesn’t appear to be legal - all outstanding cmds have to be completed by the HwStorResetBus routine before it returns:<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><b><span style="font-size:10.0pt;color:#4F81BD">HwResetBus</span></b><span style="font-size:10.0pt;color:#4F81BD">
<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:1.0in;margin-bottom:.0001pt">
<span style="font-size:10.0pt;color:#4F81BD">Pointer to the miniport driver's <a href="ms-help://MS.WDK.v10.7600.091201/Storage_r/hh/Storage_r/stormini_b3051379-4caa-4502-9492-a21672cfbf0d.xml.htm">
<i><span style="color:#4F81BD;background:#3399FF">HwStorResetBus</span></i></a> routine, which is a required entry point for all miniport drivers.
<span style="background:yellow;mso-highlight:yellow">This member has the same meaning for the Storport version of the HW_INITIALIZATION_DATA structure as it does for the SCSI Port version of the structure</span>. For more information, see the
<i>HwResetBus</i> member of HW_INITIALIZATION_DATA (SCSI)</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black">and<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><i><span style="color:#4F81BD">HwScsiResetBus</span></i><span style="color:#4F81BD">
<span style="background:yellow;mso-highlight:yellow">must complete any outstanding requests</span> by calling
<b>ScsiPortCompleteRequest</b> with the <b>SrbStatus</b> value SRB_STATUS_BUS_RESET or, for individual SRBs,
<b>ScsiPortNotification</b> with this status value.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black">and<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;color:#4F81BD">The port driver pauses all device IO queues for the adapter and then calls the
<strong><span style="font-family:"Calibri","sans-serif"">HwStorResetBus</span></strong> routine at IRQL DISPATCH_LEVEL after acquiring the StartIo spin lock.
<span style="background:yellow;mso-highlight:yellow">A miniport driver is responsible for completing SRBs received by
<a href="http://msdn.microsoft.com/en-us/library/windows/hardware/ff557423(v=vs.85).aspx">
<strong><span style="font-family:"Calibri","sans-serif";color:#4F81BD;text-decoration:none">HwStorStartIo</span></strong></a> for
<em><span style="font-family:"Calibri","sans-serif"">PathId</span></em> during this routine and setting their status to SRB_STATUS_BUS_RESET if necessary</span><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;color:#4F81BD"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black">Since HwStorResetBus must finish its work before returning; it can’t schedule a DPC to do so later on. The logic which schedules a DPC should be removed.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="color:black"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:black">Code should be added to call
<b>StorPortPause() </b>to hold off any new requests<b> </b>till <b>StorPortResume()
</b>is called.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="color:black"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:black">Code should be added to call <b>StorPortSynchronizeAccess</b>() in order to synchronize with HwStorInterrupt. A callback routine in the NVMe driver should also be added for NVMeResetBus to do the synchronized
work in. HwStorResetBus is already synchronized with HwStorStartIo since the port driver calls it only after acquiring the StartIo spinlock.
<o:p></o:p></span></p>
<p class="MsoListParagraph"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="color:black"><span style="mso-list:Ignore">4.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:black">We should implement a <b>
driver-internal global (per “adapter”) flag</b> signifying we are busy with reset processing and thus can’t allow new I/O requests to go through to the hardware.
<o:p></o:p></span></p>
<p class="MsoListParagraph"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="color:black"><span style="mso-list:Ignore">5.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:black">Code should be added to call StorPortResume() when all work is complete.
<o:p></o:p></span></p>
<p class="MsoListParagraph"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l2 level1 lfo3"><![if !supportLists]><span style="color:black"><span style="mso-list:Ignore">6.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="color:black">We should refer to the WDK-supplied LSI parallel SCSI StorPort miniport sample driver for an example of all of the above.<o:p></o:p></span></p>
<p class="MsoListParagraph"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="color:black">Judy<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>