<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style id="mysingle_style">v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<title>Samsung Enterprise Portal mySingle</title>
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";
mso-believe-normal-left:yes;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin-top:3.75pt;
margin-right:0in;
margin-bottom:3.75pt;
margin-left:0in;
font-size:9.0pt;
font-family:"Arial","sans-serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:.5in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><![if mso 9]><style>p.MsoNormal
{margin-left:7.5pt;}
</style><![endif]><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="margin-left:7.5pt;margin-top:7.5pt;margin-right:7.5pt;margin-bottom:7.5pt">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Dear all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Please see the reply from Suman and provide your thoughts.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> SUMAN PRAKASH B [mailto:suman.p@samsung.com]
<br>
<b>Sent:</b> Wednesday, November 05, 2014 3:55 AM<br>
<b>To:</b> Alex Chang; nvmewin@lists.openfabrics.org; judy.brock@ssi.samsung.com<br>
<b>Subject:</b> Re: RE: Samsung patch for Hot plug fixes<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hi Alex,</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">1. In NVMeInitCallback function, why we don’t need to check SC and SCT anymore for NVMeWaitOnLearnMapping case?</span><o:p></o:p></p>
<p><strong><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">[Suman]</span></strong><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> As Judy has mentioned, the namespace may not be ready early on during initialization. We have
tested with different devices and observed that one of the device returns NAMEPSACE_NOT_READY. And since this is not a fatal error, we can ignore checking the ST and SCT bits. Also when device returns NAMESPACE_NOT_READY, driver does not complete the learning
cores and this may result in performance degradation as context switch will happen when device interrupts.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">2. The resolution of start surprise removal timer is set as 1 second. What are the reasons to set it as 1 second rather than others?</span><o:p></o:p></p>
<p><strong><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">[Suman]</span></strong><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Generally, reading device register in a PCIe SSD is costy and can degrade the performance.
When we implemented this logic, there was no impact in performance when driver reads device register frequently, may be because the register read was in a separate thread and not in IO path. But still we did not want to keep reading the register very frequently
and also when we hot remove the device during IO, device should not take more time to be removed from device manager. Hence 1 second delay was a trade off between "not reading device register frequently" and "remove the device from device manager as soon as
possible".</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks,</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Suman</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p>------- <b>Original Message</b> -------<o:p></o:p></p>
<p><b>Sender</b> : Alex Chang<Alex.Chang@pmcs.com><o:p></o:p></p>
<p><b>Date</b> : Nov 05, 2014 01:41 (GMT+05:00)<o:p></o:p></p>
<p><b>Title</b> : RE: Samsung patch for Hot plug fixes<o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:9.0pt;font-family:"Arial","sans-serif""><!--[if mso 9]-->
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Suman,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I have couple of questions for you:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-right:7.5pt;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">1.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In NVMeInitCallback function, why we don’t need to check SC and SCT anymore for NVMeWaitOnLearnMapping case?<o:p></o:p></span></p>
<p class="MsoListParagraph" style="margin-right:7.5pt;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The resolution of start surprise removal timer is set as 1 second. What are the reasons to set it as 1 second rather than others?<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-right:15.0pt"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank you!<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-right:15.0pt"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in;border-image: none">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> SUMAN PRAKASH B [mailto:suman.p@samsung.com]
<br>
<b>Sent:</b> Tuesday, November 04, 2014 2:50 AM<br>
<b>To:</b> nvmewin@lists.openfabrics.org; Alex Chang; judy.brock@ssi.samsung.com<br>
<b>Subject:</b> Samsung patch for Hot plug fixes<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<pre>Content-Type: text/plain; charset=UTF-8<o:p></o:p></pre>
<pre>Content-Transfer-Encoding: 8bit<o:p></o:p></pre>
<pre>Date: %%SENT_DATE%%<o:p></o:p></pre>
<pre>Subject: Suspect Message Quarantined <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>%%DESC%%<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The full message and the attachment have been stored in the quarantine.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The identifier for this message is '%%QID%%'.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Access the quarantine at: <o:p></o:p></pre>
<pre>https://puremessage.pmc-sierra.bc.ca:28443/<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>For more information on PMC's Anti-Spam system:<o:p></o:p></pre>
<pre>http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>IT Services<o:p></o:p></pre>
<pre>PureMessage Admin <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Hi Everyone,</span><o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
We have a patch for the Hot plug fixes.</span><o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Please find attached the source code. The password is samsung123</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Please find the change description below -
</span></strong><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">1) Surprise removal while IOs are in progress.</span></strong><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">To reproduce this scenario -</span></strong><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Connect the disk and execute IOmeter on the disk volume. When IOs are in progress, surprise remove the device. User expects that the device should be removed from device manager immediately
and iometer should increase the error count field. This does not happen since we don't handle this scenario in OFA driver.</span><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Resolution -</span></strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
a. Added a new function IsdeviceRemoved(). This is a recursive function. Compares the values of Version Register values with old value and incase of mismatch complete the outstanding commands with SRB_STATUS_ERROR. (nvmeStd.c/IsDeviceRemoved)<br>
b. Start the Timer for IsDeviceRemoved() when the NextDriverState is set to StartComplete.(nvmeStat.c/NVMeRunning)<br>
c. Stop the timer for IsDeviceRemoved() incase of ScsiStopAdapter. (nvmeStat.c/NVMeAdapterControl)<br>
d. Restart the timer for IsDeviceRemoved() incase of ScsiRestartAdapter. (nvmeStat.c/NVMeAdapterControl)<br>
e. Stop the timer for IsDeviceRemoved() incase of SRB_FUNCTION_SHUTDOWN. (nvmeStd.c/NVMeBuildIo)<br>
f. If DeviceRemovedDuringIO flag is set to TRUE, complete the SRBs with SRB_STATUS_ERROR for the IOs. This case is to handle the IOs received once the device has been surprise removed. (nvmeStdc/NVMeBuildIo)<br>
g. Modified the prototype of NVMeDetectPendingCmds function. When device is surprise removed when IOs are pending, the outstanding IOs has to be completed with SRB_STATUS_ERROR. (nvmeIo.c/NVMeDetectPendingCmds)<br>
h. Call the NVMeDetectPendingCmds function with SRB_STATUS_BUS_RESET. (nvmeInit.c/NVMeNormalShutdown, nvmePwrMgmt.c/NVMeAdapterControlPowerDown, nvmeStd.c/RecoveryDpcRoutine)</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">2) Memory leak issues.</span></strong><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">To reproduce this scenario -</span></strong><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">a. Memory leak observed during hot removal in Resource monitor->Non-paged pool. (On Server2012R2 -> Task Manager -> Performance -> Non-paged pool)<br>
b. Memory leak observed during disable/enable the NVMe controller in device manager.</span><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Resolution -</span></strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
To fix memory leak, in NVMeBuildIo()->SRB_FUNCTION_PNP, when PnPAction is StorRemoveDevice(disable controller) and StorSurpriseRemoval(hot remove device), NVMeAdapterControlPowerDown() is invoked to stop the adapter and then NVMeFreeBuffers is invoked to free
the memory. At this point, since the ShutdownInProgress is set in NVMeAdapterControlPowerDown(), nothing is done during NVMeAdapterControl() - ScsiStopAdapter -> NVMeAdapterControlPowerDown().</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">3) Surprise Removal during Disk Initialization</span></strong><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">To reproduce this scenario -
</span></strong><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Hot insert the device and hot remove the device immediately. At this point, our driver might be executing the initialization state machine in NVMePassiveInitialize. The device will
not be immediately removed from the device manger. The while loop will be active till passiveTimeout happens, then system BSOD.</span><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Resolution -</span></strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
a. Read the Version register. This is used to compare against the value in version register after a surprise removal. (nvmeStd.c/NVMeFindAdapter)<br>
b. Read the Version Register and compare with old Version Register value(i.e. value read in NVMeFindAdapter). Mismatch in these values means surprise removal. (nvmeStd.c/NVMePassiveInitialize)<br>
c. Set the NextDriverState to NVMeStartComplete and DeviceRemovedDuringIO to TRUE and return TRUE from NVMePassiveInitialize.<br>
d. Driver may get commands in NVMeBuildIo, where driver returns SRB_STATUS_ERROR when DeviceRemovedDuringIO is set to TRUE.<br>
e. Then NVMeAdapterControl() - ScsiStopAdapter is executed. </span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">4) Delay in removing the device from device manager after hot removal of device.
</span></strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">When device is hot removed, the NVMeAdapterControlPowerDown() -> NVMeResetAdapter() -> NVMeWaitForCtrlRDY() is invoked which sets
</span><o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">the EN bit to 0 and waits for RDY bit to become 0. Since the device is physically removed, the memory mapped registers will be come all 1's and the RDY bit will never become 0. Hence the while
loop in NVMeWaitForCtrlRDY() is active for some time even after device removal and hence device is not removed from device manager immediately.
</span><o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Resolution -</span></strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
Check for the value of CSTS. If its 0xFFFFFFFF, then device has been surprise removed and return TRUE. (nvmeStd.c/NVMeWaitForCtrlRDY)</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">5) Avoid redundant call of NVMeResetAdapter()</span></strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
a. File/Function: nvmeInit.c/NVMeEnableAdapter - Removed the NVMeResetAdapter() function call from NVMeEnableAdapter() as this is redundant. The NVMeResetAdapter() is being invoked in the RecoveryDpcRouitne() and then again its being invoked in the NVMeEnableAdapter.<br>
b. In the NVMeInitialize() function the EN and RDY bit are set to 0 before the NVMeEnableAdapter() is being invoked. But NVMeResetAdapter() does again the same functionality.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">6) </span>
</strong><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">When testing hot insertion with different devices, we observed some devices returned NAMESPACE_NOT_READY for IO commands during learning cores and disk initialization(report luns, inquiry,
etc). To address this issue and provide support for these devices in the driver, we have done the following changes.<br>
a. During learning cores, driver sends read commands on all the queues to get the core to MSI-x mapping. When the read commands are interrupted, in the NVMeInitCallback(), if the SC and SCT values are not 0, then the learning cores is not completed. This check
is not required as driver wants only the core to MSI-x mapping. Since this is not a fatal error, we can skip reading the SC and SCT values, as this will impact the performance. (nvmeInit.c/NVMeInitCallback).<br>
b. Following the above, when the initialization state machine is complete and kernel starts sending SCSI commands for disk initialization, and when device returns NAMESPACE_NOT_READY, this has to be translated to the corresponding SCSI sense data so that the
commands will be re-tried after some time. (nvmeSnti.c/genericCommandStatusTable[]).
</span><o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif""><br>
<strong><span style="font-family:"Calibri","sans-serif"">Tested the following.</span></strong></span><o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">- WHCK on Win7 and 2012R2<br>
- Install/Uninstall, Enable/Disable, FS Format<br>
- Hibernation/Resume, Sleep/Resume<br>
- IOmeter<br>
- Hot removal which iometer is running.<br>
- Hot removal immediately after hot insertion.<br>
- Continous hot insert and remove operations.<br>
- Check for device removal after following sequence - Hot insert, system hibernation, Hot remove, system resume.<br>
- Check for device presense after following sequence - System hibernation, hot insert, system resume.<br>
- Memory leaks during hot plug operations and disable/enable.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Thanks,<br>
Suman</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellpadding="0" id="confidentialsignimg">
<tbody>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p><img width="520" height="144" id="_x0000_i1025" src="cid:T9SZN3WZA6X7@namo.co.kr"><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="border:solid windowtext 1.0pt;padding:0in"><img width="100" height="100" id="_x0000_i1026" src="cid:~WRD176.jpg" alt="Image removed by sender."></span><o:p></o:p></p>
</div>
</body>
</html>