<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<title>Samsung Enterprise Portal mySingle</title>
<style><!--
/* Font Definitions */
@font-face
{font-family:Gulim;
panose-1:2 11 6 0 0 1 1 1 1 1;}
@font-face
{font-family:Gulim;
panose-1:2 11 6 0 0 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@Gulim";
panose-1:2 11 6 0 0 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Gulim","sans-serif";
mso-believe-normal-left:yes;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin-top:3.0pt;
margin-right:0in;
margin-bottom:3.0pt;
margin-left:0in;
font-size:9.0pt;
font-family:"Arial","sans-serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Times New Roman","serif";
color:#1F497D;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><![if mso 9]><style>p.MsoNormal
{margin-left:6.0pt;}
</style><![endif]><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="margin-left:6.0pt;margin-top:6.0pt;margin-right:6.0pt;margin-bottom:6.0pt">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">Hello Judy/Wonmoon,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">Sorry for the late response. The “hot remove” state is just a state that we enter when the driver receives a Format command. Basically, this state will
remove the namespace(s) from the topology by calling StorPortNotification() with BusChangeDetected. This will remove the “SCSI target/disk” associated with each namespace form the OS (because Storport will re-enumerate the controller and the driver will not
expose the namespaces about to be formatted) so that the format can occur on the relevant namespaces.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">By signaling Windows that the namespaces have been “removed”, all I/O will be stopped by the OS. Then the format can complete. Once the format is complete,
we perform the opposite action to “hot add” the namespace back into the topology by calling StorPortNotification() with BusChangeDetected… only this time, we will surface the namespace(s) again when Storport re-enumerates.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">Not queues are deleted in this state, no memory is de-allocated, and nothing else changes about the namespace. This is simply the first step (in a 3 step
sequence) when formatting a namespace as we cannot format a namespace while the OS is aware of its presence and could be potentially sending I/O to a stale namespace config (i.e. changing LBA/sector size).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">Let me know if this answers your question.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D">Ray<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Times New Roman","serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Judy Brock-SSI [mailto:judy.brock@ssi.samsung.com]
<br>
<b>Sent:</b> Wednesday, April 17, 2013 6:21 AM<br>
<b>To:</b> WONMOON CHEON; </span><span style="font-size:10.0pt">강미경</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">; technical@nvmexpress.org<br>
<b>Subject:</b> RE: RE: Handling pending commands when processing Format<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> >>Would you elaborate more about the "hot-remove" state? In this state, do you mean that all the IO SQ/CQs are deleted? Or, waiting for completions of all the outstanding IOs?</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The IO SQ/CQs are definitely NOT deleted. I would need to look more closely through the driver code to see how IOs previously sent to the namespaces which are
marked as OFFLINE are handled/finished/quiesced. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">There are other folks on this thread who no doubt have more history/intimate knowledge of this driver than I do who may answer that question more quickly than
I can…also, perhaps this discussion should probably be moved to the OFA driver forum since it has turned into a driver-specific thread at this point.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">What do folks think?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Judy<o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
</span><span style="font-size:10.0pt">천원문</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> [<a href="mailto:wm.cheon@samsung.com">mailto:wm.cheon@samsung.com</a>]
<br>
<b>Sent:</b> Wednesday, April 17, 2013 1:20 AM<br>
<b>To:</b> Judy Brock-SSI; </span><span style="font-size:10.0pt">강미경</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">;
<a href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</a><br>
<b>Subject:</b> Re: RE: Handling pending commands when processing Format<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hi Judy,</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Would you elaborate more about the "hot-remove" state? In this state, do you mean that all the IO SQ/CQs are deleted? Or, waiting for completions of all the outstanding IOs?</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks,</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Wonmoon</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p>------- <b>Original Message</b> -------<o:p></o:p></p>
<p><b>Sender</b> : Judy Brock-SSI<<a href="mailto:judy.brock@ssi.samsung.com">judy.brock@ssi.samsung.com</a>><o:p></o:p></p>
<p><b>Date</b> : 2013-04-17 16:39 (GMT+09:00)<o:p></o:p></p>
<p><b>Title</b> : RE: Handling pending commands when processing Format<o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I should clarify that it is not the Windows operating system – but rather the Windows OFA NVMe driver - that, from what I can see, does a “hot-remove” of
all namespace(s) associated with a device before allowing a format operation to begin; “hot remove” is just the name for an internal state in the driver format nvm state machine.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Before beginning the actual format op, the driver internally marks all namespaces associated with the format operation “offline”. It then notifies the OS that
there has been a “bus change” event (via an OS-specific API).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">This in turn will cause the OS to rescan (re-enumerate) the “bus” (the pseudo SCSI bus, that is – we expose NVM namespaces as SCSI luns).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Since all the pertinent namespaces have been marked offline internally, the bus rescan won’t detect any valid SCSI luns (because the driver will not report
any).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hence from the OS point of view, any SCSI lun(s) previously mapped to the namespace(s) to be formatted will have disappeared/will be unaddressable while the
format operation is in progress.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Judy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Judy Brock-SSI
<br>
<b>Sent:</b> Tuesday, April 16, 2013 8:22 PM<br>
<b>To:</b> 'mkkang.kang@samsung.com'; <a href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</a><br>
<b>Subject:</b> RE: Handling pending commands when processing Format<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Mikyeong,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I haven’t looked at the Linux driver but I know that Windows hot-removes all namespace(s) associated with a device before allowing a format operation to begin.
And a namespace can’t be removed while there is IO outstanding to it so that answers your question regarding IOs being completed before format begins. It also answers the question about requests being sent to a namespace while format is in progress – can’t
happen.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Judy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
</span><span lang="KO" style="font-size:10.0pt;mso-fareast-language:KO">강미경</span><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> [<a href="mailto:mkkang.kang@samsung.com">mailto:mkkang.kang@samsung.com</a>]
<br>
<b>Sent:</b> Tuesday, April 16, 2013 7:07 PM<br>
<b>To:</b> <a href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</a><br>
<b>Subject:</b> Handling pending commands when processing Format<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Dear All,</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Format NVM command may change the Namespace repository, and it will be executed out of order like any other commands. Therefore, Format NVM command may affect other commands that are pending
execution in the device, if any. </span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">1) How does an OFA/linux driver handle 'Format NVM command'? Does a host make sure that all commands for a particular NSID are completed before sending 'Format NVM command'?</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">2) If a host driver does not behave like 1) above, how can a device handle other pending commands which were previously submitted in a SQ? It seems like we need an additional status code. e.g.
Abort due to Namespace Format</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">3) Let's suppose that 'Format NVM command' is in progress. If the host driver sends subsequent commands to the namespace being formatted, should the device reply directly with a 'Namespace
not Ready'?</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">[1.0e spec] If the device does not reply directly and the format operation takes long time, then, I/O command will timeout and the host may send the reset. But if commands are responded with
'Namespace Not Ready', host may not issue the reset. Therefore, direct reply seems to be needed.
</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">[1.1 spec. ECN 001] There is Format progress indicator. The host driver can check format progress any time, therefore, there is no concern about reset during format command.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Best Regards,</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Mikyeong Kang</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<div>
<div>
<div>
<div class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
<hr size="3" width="240" style="width:2.5in" align="left">
</span></div>
</div>
</div>
</div>
<p><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Kang MiKyeong</span></b><o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black">Flash Memory Planning/Enabling Group, Memory Div.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black;background:white">SAMSUNG ELECTRONICS, Co., Ltd..</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black;background:white">Phone: 82-31-208-3857</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">Mobile: 82-10-9369-0177</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black;background:white">E-mail:
<a href="mailto:mkkang.kang@samsung.com">mkkang.kang@samsung.com</a></span><o:p></o:p></p>
<div>
<div>
<div>
<div class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Calibri","sans-serif"">
<hr size="3" width="240" style="width:2.5in" align="left">
</span></div>
</div>
</div>
</div>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="3" cellpadding="0" id="confidentialsignimg">
<tbody>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p><img border="0" width="520" height="144" id="_x0000_i1027" src="cid:image001.gif@01CE3C43.8279D160"><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><img border="0" id="_x0000_i1028" src="http://ext.samsung.net/mailcheck/SeenTimeChecker?do=dd90ab357fccaa0879ef49d135999623709a3a8db170c13642590acae5bdcb6a1b41a3ea5b94b2ed546e5f4bdee86ec72cb0372888719872db9fdddda33e82cbe4a391424e62fcf6cf878f9a26ce15a0"><o:p></o:p></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-family:"Times New Roman","serif""><img border="0" id="_x0000_i1029" src="http://ext.samsung.net/mailcheck/SeenTimeChecker?do=dd90ab357fccaa0810067d8e81f72583c0a10306535965788c7e8eca81cb564b226b8d944a9d91d2f73afb615c53a27ef4bcdeced46ed5ee08cece8541bc14eacf878f9a26ce15a0"><o:p></o:p></span></p>
</div>
</body>
</html>