<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<title>Samsung Enterprise Portal mySingle</title>
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-believe-normal-left:yes;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin-top:3.75pt;
margin-right:0in;
margin-bottom:3.75pt;
margin-left:0in;
font-size:9.0pt;
font-family:"Arial",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#002060;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><![if mso 9]><style>p.MsoNormal
{margin-left:7.5pt;}
</style><![endif]><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="margin-left:7.5pt;margin-top:7.5pt;margin-right:7.5pt;margin-bottom:7.5pt">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#002060">Hi Suman… Intel has no comments/concerns. Everything looks good. I’ll push the patch in the next day.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#002060">I believe Samsung had one more patch for concurrent channels, correct? Was there anything after that? I’d like to start planning for the mid-year release.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#002060">Thank you… Ray<o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> SUMAN PRAKASH B [mailto:suman.p@samsung.com]
<br>
<b>Sent:</b> Monday, May 30, 2016 8:00 AM<br>
<b>To:</b> Thomas Freeman <thomas.freeman@hgst.com>; nvmewin@lists.openfabrics.org; Robles, Raymond C <raymond.c.robles@intel.com><br>
<b>Subject:</b> Re: RE: [nvmewin] Patch for handling surprise removal in IOCTL path<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Thanks Tom.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">To other reviewers: Please let me know if there is any comments/concerns.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">- Suman</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p>------- <b>Original Message</b> -------<o:p></o:p></p>
<p><b>Sender</b> : Thomas Freeman<<a href="mailto:thomas.freeman@hgst.com">thomas.freeman@hgst.com</a>><o:p></o:p></p>
<p><b>Date</b> : May 27, 2016 02:45 (GMT+05:30)<o:p></o:p></p>
<p><b>Title</b> : RE: [nvmewin] Patch for handling surprise removal in IOCTL path<o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Suman,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I’ve reviewed changes and have no comments/concerns.<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Thank you,<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">Tom Freeman<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">Software Engineer, Device Manager and Driver Development<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">Western Digital Corporation<o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">e.
<a href="mailto:Thomas.freeman@hgst.com">Thomas.freeman@hgst.com</a><o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">o. +1-507-322-2311<o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> nvmewin [<a href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</a>]
<b>On Behalf Of </b>SUMAN PRAKASH B<br>
<b>Sent:</b> Monday, May 16, 2016 8:21 AM<br>
<b>To:</b> <a href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</a>;
<a href="mailto:raymond.c.robles@intel.com">raymond.c.robles@intel.com</a><br>
<b>Subject:</b> [nvmewin] Patch for handling surprise removal in IOCTL path<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Hi All,</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">This patch includes changes for supporting device surprise removal in IOCTL path. Samsung already submitted a patch to handle surprise removal during normal I/O in 2014. This patch is an extention
to the previous patch.</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">I have made a detailed overview of the changes in the attached doc file(the contents are also copied here below) and the attached zip file contains the source code.</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Password is samsungnvme<br>
<br>
Please let me know if you have any questions.<br>
<br>
Thanks,<br>
Suman<br>
<br>
***************************************</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Handling device Surprise removal in IOCTL path:</span></u></strong><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Current OFA driver does not need any change to support surprise removal when no I/Os are outstanding. But when I/Os are outstanding and the device is surprise removed, the tool(lets say
IOMeter) hangs and the device is not removed from the device manager for very long time. Since NVMe is suprise remove capable device, user expectation is that the tool should exit/stop gracefully and the device should be removed from the device manage immediately.</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> The Windows storage stack currently does not handle surprise removal during outstanding I/Os. To handle this scenario, we have submitted a patch in 2014. But this patch was handling surprise
removal during normal I/O commands and not when IOCTL commands from proprietary tools were outstanding.<br>
In the previous patch, we had implemented the StorPortNotification(RequestTimerCall,…) to handle surprise removal durign I/O. This API does not detect surprise removal when executing IOCTL commands. For example, when executing Format NVM command with Secure
erase or Cryptographic erase, when device is surprise removed, the device is not removed from the device manager and system hangs.</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> To fix this issue, in Windows 8 and later versions, there is support for StorPortRequestTimer() which does the same as StorPortNotification(RequestTimerCall,…). When StorPortRequestTimer()
is used, device surprise removal during IOCTL commands is also detected. Hence for Win 7 kernel, StorPortNotification(RequestTimerCall,…) and for Win 8 and above kernels, StorPortRequestTimer() can be used.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Code changes:</span></u></strong><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">1. In NVMeFindAdapter(), initialize the timer routine using StorPortInitializeTimer() API.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">2. In NVMeRunning(), in NVMeStartComplete, start the surprise removal timer routine.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">3. Stop the timer routine using StorPortRequestTimer() in following functions:<br>
a. In NVMeAdapterControl(), in ScsiStopAdapter, when shutting down the device<br>
b. In NVMeBuildIo(), for SRB_FUNCTION_SHUTDOWN<br>
c. In NVMeBuildIo(), for SRB_FUNCTION_PNP, if PnPAction is StorRemoveDevice or StorSurpriseRemoval and ntldrDump is FALSE</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">4. Free the timer routine using StorPortFreeTimer() in following functions:<br>
a. In NVMePassiveInitialize(), when the initialization state machine fails<br>
b. In NVMeAdapterControl(), in ScsiStopAdapter, when shutting down the device<br>
c. In NVMeBuildIo(), for SRB_FUNCTION_SHUTDOWN,<br>
d. In NVMeBuildIo(), for SRB_FUNCTION_PNP, if PnPAction is StorRemoveDevice or StorSurpriseRemoval and ntldrDump is FALSE<br>
e. In NVMeStartIo(), when NextDriverState is not NVMeStartComplete<br>
f. In RecoveryDpcRoutine()</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">5. New Surprise removal timer routine IsDeviceRemoved() for kernels above Windows 8 which uses StorPortFreeTimer() and StorPortRequestTimer().</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">6. In FormatNVMGetIdentify(), when the structure to retrieve is Identify namespace, the lunId is set to INVALID_LUN_EXTN, because of which BSOD is observed. Invoked the function NVMeIsNamespaceVisible()
to correct this. We observed this when secure erase in IOCTL is in progress, and device is surprise removed.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">7. In NVMeBuildIo(), to block read/write commands when format nvm is in progress, we have a check. But when formatNVM is in progress, we get requests other than SRB_FUNCTION_EXECUTE_SCSI, like
SRB_FUNCTION_PNP or SRB_FUNCTION_IO_CONTROL, in which case opCode = GET_OPCODE(Srb) in line number 1125 results in BSOD. This is observed when secure erase is executed in IOCTL path, and system is hibernated and then device is surprise removed. Hence moved
the blocking of Read/Write commands while Format NVM is in progress code inside SRB_FUNCTION_EXECUTE_SCSI.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">8. Changes in inf file:<br>
a. Following line has been included to allow interrupts on processors beyond group 0. For the StorPortInitializePerfOpts() MessageTargets to work correctly when a system has multiple groups are, the following line is required, otherwise we observe BSOD.
Also, when we tested in a system which has 1 group by default and 32 logical processors in Server 2012R2, some WHQL tests create multiple groups after which the driver does not work properly if the below line is not included in inf file.<br>
HKR, Interrupt Management\Affinity Policy, GroupPolicy, %REG_DWORD%, 1</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> b. Following line has been included to specifies that the device's interrupts are of high priority.<br>
HKR, Interrupt Management\Affinity Policy, DevicePriority, %REG_DWORD%, 3</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">9. As requested by HGST, the MAX_NAMESPACES is changed to 128. Correspondingly, the DUMP_BUFFER_SIZE is also changed from (5*64*1024) to ((5*64*1024) + (sizeof(NVME_LUN_EXTENSION)*MAX_NAMESPACES)).
If DUMP_BUFFER_SIZE is not increased, in dump mode, in NVMeInitialize(), memory allocation for pAE->pLunExtensionTable will fail.</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Also, the correct method to implement the above would be to allocate memory for pAE->pLunExtensionTable in NVMeInitCallback() in the NVMeWaitOnIdentifyCtrl state, where we get the number of
Namespaces supported by the device, and remove the memory allocated for pAE->pLunExtensionTable in dump mode(NVMeInitialize()) and in normal mode(NVMePassiveInitialize()).</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">For this patch, we have set DUMP_BUFFER_SIZE to ((5*64*1024) + (sizeof(NVME_LUN_EXTENSION) * MAX_NAMESPACES)). Please note that as per MSDN, we should not allocate more than 32 KB for RequestedDumpBufferSize.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">10. Implemented ScsiAdapterPower controltype in NVMeAdapterControl(). ScsiAdapterPower executes in <= DISPATCH_IRQL. This is supported only from Windows 8. Since we need to free the buffer and
stop the timer only at IRQL <= DISPATCH_IRQL, this is implemented. Also, this will avoid executing initialization state machine in NVMeReInitializeController in ScsiRestartAdapter(DIRQL) during resume from hibernation.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">We have tested following:</span></u></strong><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">a. WHQL tests</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">b. Connect device -> execute secure erase in IOCTL path -> during secure erase execution remove device -> expectation: disk should be removed and tool(executing the secure erase) should exit
gracefully</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">c. Connect device -> hibernation -> remove device -> resume -> expectation: disk should be removed</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">d. Connect device -> run IOMeter -> execute secure erase in IOCTL path -> hibernation -> remove device -> resume -> expectation: disk should be removed and tools should exit gracefully</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">c. OS installation</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><img border="0" id="_x0000_i1025" src="http://ext.samsung.net/mailcheck/SeenTimeChecker?do=e7bc4897eaf7ac7a189c3eecb549349dcf4c3e2988f12776313cb48408a29da1bbef98290b9ea6844a835b19b8ec809fc7b41e955949e5c8a728c55b39cc59eacf878f9a26ce15a0"><o:p></o:p></p>
</div>
</body>
</html>