<HTML xmlns:v = "urn:schemas-microsoft-com:vml" xmlns:o = "urn:schemas-microsoft-com:office:office" xmlns:w = "urn:schemas-microsoft-com:office:word" xmlns:x = "urn:schemas-microsoft-com:office:excel" xmlns:m = "http://schemas.microsoft.com/office/2004/12/omml"><HEAD><TITLE>Samsung Enterprise Portal mySingle</TITLE>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<STYLE id=mysingle_style type=text/css>P {
MARGIN-BOTTOM: 5px; FONT-SIZE: 9pt; FONT-FAMILY: Arial, arial; MARGIN-TOP: 5px
}
TD {
MARGIN-BOTTOM: 5px; FONT-SIZE: 9pt; FONT-FAMILY: Arial, arial; MARGIN-TOP: 5px
}
LI {
MARGIN-BOTTOM: 5px; FONT-SIZE: 9pt; FONT-FAMILY: Arial, arial; MARGIN-TOP: 5px
}
BODY {
FONT-SIZE: 9pt; FONT-FAMILY: Arial, arial; MARGIN: 10px; LINE-HEIGHT: 1.4
}
</STYLE>
<META content=IE=5 http-equiv=X-UA-Compatible>
<META content=IE=5 http-equiv=X-UA-Compatible>
<META name=GENERATOR content="MSHTML 11.00.9600.18212"></HEAD>
<BODY>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri">Hi All,</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri">I am sending the updated patch incorporating feedback from Carolyn. The changes are listed below. The password is samsungnvme</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri">1. In NVMeRunningWaitOnLearnMapping(), if there is zero namespace it is not necessary to go to NVMeWaitOnNamespaceReady , instead we can directly start NVMeStartComplete.<BR>2. In NVMeRunningWaitOnNamespaceReady(), for crash/Hibernate mode it is not necessary to go to the NVMeWaitOnNamespaceReady.</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri">Please let me know if you have any questions.</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri">Thanks,<BR>Suman</SPAN></P>
<P> </P>
<P>------- <B>Original Message</B> -------</P>
<P><B>Sender</B> : SUMAN PRAKASH B<suman.p@samsung.com> Senior Chief Engineer/SSIR-SSD Solutions/Samsung Electronics</P>
<P><B>Date</B> : Apr 11, 2016 21:22 (GMT+05:30)</P>
<P><B>Title</B> : Re: RE: RE: [nvmewin] Patch with changes for Optimizing disk initialization performance</P>
<P> </P>
<META content=IE=5 http-equiv=X-UA-Compatible>
<META content=IE=5 http-equiv=X-UA-Compatible>
<META name=GENERATOR content=ActiveSquare>
<META content=IE=5 http-equiv=X-UA-Compatible><X-BODY>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">Hi Carolyn,</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">Thanks for testing our patch.</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">We tried the following:</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">a. Sequential read performance on single worker, single queue depth, on system with 32 cores, and did not observe performance drop compared to R133.</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">b. Assuming that the IOs are getting scattered across multiple NUMA nodes and due to remote memory access, the 10% drop could have been observed, we affinitized the performance tool on NUMA node 1, assuming that the IO queue is created on NUMA node 0. After affinitization, application submits IO on node 1 and driver processes the IOs in node 0. But still, we did not observe any performance drop.</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000"></SPAN> </P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">As there are no major issues here, </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">we will incorporate the following review comments and share the revised patch tomorrow:</SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">a.</SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000"> </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">nvmeStat.c @ line 784 : if there is zero namespace it is not necessary to go to NVMeWaitOnNamespaceReady , instead we can directly start NVMeStartComplete.</SPAN></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: rgb(31,73,125)'></SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">b. </SPAN><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">nvmeStat.c @line 899 : In crash/Hibernate mode it is not necessary to go to the NVMeWaitOnNamespaceReady.</SPAN></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000"></SPAN></o:p> </P></SPAN>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">Thanks,</SPAN></P>
<P><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Calibri; COLOR: #000000">Suman</SPAN></P>
<P> </P>
<P>------- <B>Original Message</B> -------</P>
<P><B>Sender</B> : Foster, Carolyn D<carolyn.d.foster@intel.com></P>
<P><B>Date</B> : Apr 08, 2016 02:42 (GMT+05:30)</P>
<P><B>Title</B> : RE: RE: [nvmewin] Patch with changes for Optimizing disk initialization performance</P>
<P> </P>
<META name=Generator content=ActiveSquare>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<STYLE>@font-face {
font-family: Cambria Math;
}
@font-face {
font-family: Calibri;
}
@font-face {
font-family: Verdana;
}
@page WordSection1 {size: 8.5in 11.0in; margin: 1.0in 1.0in 1.0in 1.0in; }
P.MsoNormal {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-believe-normal-left: yes
}
LI.MsoNormal {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-believe-normal-left: yes
}
DIV.MsoNormal {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-believe-normal-left: yes
}
A:link {
TEXT-DECORATION: underline; COLOR: #0563c1; mso-style-priority: 99
}
SPAN.MsoHyperlink {
TEXT-DECORATION: underline; COLOR: #0563c1; mso-style-priority: 99
}
A:visited {
TEXT-DECORATION: underline; COLOR: #954f72; mso-style-priority: 99
}
SPAN.MsoHyperlinkFollowed {
TEXT-DECORATION: underline; COLOR: #954f72; mso-style-priority: 99
}
P {
FONT-SIZE: 9pt; FONT-FAMILY: "Arial",sans-serif; MARGIN: 3.75pt 0in; mso-style-priority: 99
}
P.MsoNoSpacing {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 1
}
LI.MsoNoSpacing {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 1
}
DIV.MsoNoSpacing {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 1
}
P.MsoListParagraph {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0.5in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 34
}
LI.MsoListParagraph {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0.5in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 34
}
DIV.MsoListParagraph {
FONT-SIZE: 12pt; FONT-FAMILY: "Times New Roman",serif; MARGIN-LEFT: 0.5in; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 34
}
P.p1 {
FONT-SIZE: 9pt; FONT-FAMILY: "Arial",sans-serif; MARGIN: 3.75pt 0in; mso-style-priority: 99; mso-style-name: p1
}
LI.p1 {
FONT-SIZE: 9pt; FONT-FAMILY: "Arial",sans-serif; MARGIN: 3.75pt 0in; mso-style-priority: 99; mso-style-name: p1
}
DIV.p1 {
FONT-SIZE: 9pt; FONT-FAMILY: "Arial",sans-serif; MARGIN: 3.75pt 0in; mso-style-priority: 99; mso-style-name: p1
}
SPAN.apple-style-span {
mso-style-name: apple-style-span
}
SPAN.EmailStyle22 {
FONT-FAMILY: "Calibri",sans-serif; FONT-WEIGHT: normal; COLOR: #1f497d; FONT-STYLE: normal; mso-style-type: personal
}
SPAN.EmailStyle23 {
FONT-FAMILY: "Calibri",sans-serif; FONT-WEIGHT: normal; COLOR: windowtext; FONT-STYLE: normal; mso-style-type: personal
}
SPAN.s1 {
mso-style-name: s1
}
SPAN.EmailStyle25 {
FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d; mso-style-type: personal
}
SPAN.EmailStyle26 {
FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d; mso-style-type: personal
}
SPAN.EmailStyle28 {
FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d; mso-style-type: personal-reply
}
.MsoChpDefault {
FONT-SIZE: 10pt; mso-style-type: export-only
}
DIV.WordSection1 {
page: WordSection1
}
</STYLE>
<META content=IE=5 http-equiv=X-UA-Compatible><X-BODY lang=EN-US style="MARGIN: 7.5pt" link="#0563C1" vlink="#954F72">
<DIV class=WordSection1>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Hi Suman,</SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>I have an update on our performance data. It appears on systems with more than 32 and more than 64 cores that there was no performance delta in the following workloads:<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>4 workers, 32 Queue Depth<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>8 workers, 16 Queue Depth<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p></o:p></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>We did observe what looks like a more consistent (and much smaller) 10% drop on workload with a single worker, single queue depth, on systems with 32 cores.<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p></o:p></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>It seems that our initial results might have been flawed and based on your comments and performance analysis, there may be no major issue here.<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p></o:p></SPAN> </P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Carolyn<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-TOP: #e1e1e1 1pt solid; BORDER-RIGHT: medium none; BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 3pt; PADDING-LEFT: 0in; BORDER-LEFT: medium none; PADDING-RIGHT: 0in">
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>From:</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'> Foster, Carolyn D <BR><B>Sent:</B> Wednesday, April 06, 2016 2:54 PM<BR><B>To:</B> 'suman.p@samsung.com' <suman.p@samsung.com>; Thomas Freeman <thomas.freeman@hgst.com>; nvmewin@lists.openfabrics.org<BR><B>Cc:</B> Seokhwan Kim <sukka.kim@samsung.com>; ANSHUL SHARMA <anshul@samsung.com>; MANOJ THAPLIYAL <m.thapliyal@samsung.com>; tru.nguyen@ssi.samsung.com<BR><B>Subject:</B> RE: RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P class=MsoNormal><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Hi Suman, thank you for the clarification. I will confirm the rest of the workload details and have that information for you tomorrow. In the mean time I will also rerun our performance tests to confirm that the results are reproducible, and will run the tests without line 594 as you suggested.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Carolyn<o:p></o:p></SPAN></P>
<DIV>
<DIV style="BORDER-TOP: #e1e1e1 1pt solid; BORDER-RIGHT: medium none; BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 3pt; PADDING-LEFT: 0in; BORDER-LEFT: medium none; PADDING-RIGHT: 0in">
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>From:</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'> SUMAN PRAKASH B [<A href="mailto:suman.p@samsung.com">mailto:suman.p@samsung.com</A>] <BR><B>Sent:</B> Wednesday, April 06, 2016 9:01 AM<BR><B>To:</B> Foster, Carolyn D <<A href="mailto:carolyn.d.foster@intel.com">carolyn.d.foster@intel.com</A>>; Thomas Freeman <<A href="mailto:thomas.freeman@hgst.com">thomas.freeman@hgst.com</A>>; <A href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</A><BR><B>Cc:</B> Seokhwan Kim <<A href="mailto:sukka.kim@samsung.com">sukka.kim@samsung.com</A>>; ANSHUL SHARMA <<A href="mailto:anshul@samsung.com">anshul@samsung.com</A>>; MANOJ THAPLIYAL <<A href="mailto:m.thapliyal@samsung.com">m.thapliyal@samsung.com</A>>; <A href="mailto:tru.nguyen@ssi.samsung.com">tru.nguyen@ssi.samsung.com</A><BR><B>Subject:</B> Re: RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Hi Carolyn,</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Thanks for the comments and suggestions. Please find my comments below:</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><STRONG><U><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>1. Observed performance degradation, potentially related to line 594 in nvmeInit.c</SPAN></U></STRONG><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><BR>a. We tested on Servers with 32 and 64 logical processors, Windows 8.1 and 2012 R2 OS, multiple vendor devices with both 1-to-1 core-queue mapping and many-to-1 core-queue mapping, with both R133 and latest drivers, and we did not observe any performance drop for 100% Sequential Read(128K) with 32 and 64 worker threads, and queue depth 32. We have tested both the StorPortInitializePerfOpts() pass case and fail case(learning cores will be executed).</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>b. Regarding MsgID-- in nvmeInit.c line number 594, NVMe device supports number of msg ids equal to 1 admin + N IO queues. But in OFA driver, since the msg id 0 is shared between admin queue and io queue, always ((1 admin + N io queues) - 1) number of msgids is used. With MsgID--, we make sure all IO queues are created with unique msg id and msg id 0 is shared with admin queue and 1 io queue. If the device that you are testing has total number of msg ids equal to 1 admin queue + N io queues, then MsgID-- should not be a problem. But if you strongly feel that MsgID-- could be an issue, can you please take the perf benchmark after removing MsgID--.</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>c. Can you please let us know the number of queues and number of Messaged IDs supported by the target device that you are testing with.</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>d. On servers, we usually test on Windows Server edition OSes. When we tested with Windows 8.1, we observed that the number of logical processors supported in Windows 8.1 is maximum 32, even when the server have more than 32 logical processors.</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>We will try to reproduce the performance degradation behavior, meanwhile <STRONG><U><SPAN style='FONT-FAMILY: "Calibri",sans-serif'>can you please provide us more debug data</SPAN></U></STRONG>.</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><STRONG><U><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>2. nvmeStat.c @ line 784 </SPAN></U></STRONG><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>: Agreed. We will change as suggested.</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><STRONG><U><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>3. nvmeStat.c @line 899</SPAN></U></STRONG><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'> : Agreed. We will change as suggested.</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Thanks,<BR>Suman</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P>------- <B>Original Message</B> -------<o:p></o:p></P>
<P><B>Sender</B> : Foster, Carolyn D<<A href="mailto:carolyn.d.foster@intel.com">carolyn.d.foster@intel.com</A>><o:p></o:p></P>
<P><B>Date</B> : Apr 06, 2016 05:33 (GMT+05:30)<o:p></o:p></P>
<P><B>Title</B> : RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></P>
<P> <o:p></o:p></P>
<P class=MsoNormalCxSpFirst style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Hi Suman,<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormalCxSpLast style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>I have a few comments and suggestions:<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpFirst style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.5in; MARGIN-RIGHT: 7.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>1.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Observed performance degradation, potentially related to line 594 in nvmeInit.c – We noticed on some systems we saw a degradation in performance and we suspect it’s related to this change. If we don’t share MSIX vector 0 between the admin queue and an IO queue we are creating one fewer queue to submit IO to. Did you execute any performance testing before and after these changes? I have included some details about the system configurations we tested and the observed results below.<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpMiddle style="MARGIN-BOTTOM: 0pt; MARGIN-LEFT: 0.5in; MARGIN-RIGHT: 0in; TEXT-INDENT: -0.25in; mso-margin-top-alt: 0in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>2.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>nvmeStat.c @ line 784 : if there is zero namespace it is not necessary to go to NVMeWaitOnNamespaceReady , instead we can directly start NVMeStartComplete.<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpLast style="MARGIN-BOTTOM: 0pt; MARGIN-LEFT: 0.5in; MARGIN-RIGHT: 0in; TEXT-INDENT: -0.25in; mso-margin-top-alt: 0in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>3.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>nvmeStat.c @line 899 : In crash/Hibernate mode it is not necessary to go to the NVMeWaitOnNamespaceReady.<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpFirst style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Performance configuration and data:<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>OS: Windows 8.1 x64<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Workload: 100% sequential Read<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Compared the OFA trunk to the Samsung patch<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 15pt; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormalCxSpLast style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 15pt; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Summary of observed results:<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpFirst style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.25in; MARGIN-RIGHT: 45pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Symbol; COLOR: #1f497d">·</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>System with fewer than 32 logical CPU cores: <o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpMiddle style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.75in; MARGIN-RIGHT: 52.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Courier New"; COLOR: #1f497d'>o</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>No delta in performance observed<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpMiddle style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.25in; MARGIN-RIGHT: 52.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Symbol; COLOR: #1f497d">·</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>System with between 32 and 64 cores: <o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpMiddle style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.75in; MARGIN-RIGHT: 22.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Courier New"; COLOR: #1f497d'>o</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>20%-50% drop in performance observed<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpMiddle style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.25in; MARGIN-RIGHT: 22.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style="FONT-SIZE: 11pt; FONT-FAMILY: Symbol; COLOR: #1f497d">·</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>System with more than 64 cores:<o:p></o:p></SPAN></P>
<P class=MsoListParagraphCxSpLast style="MARGIN-BOTTOM: 7.5pt; MARGIN-LEFT: 0.75in; MARGIN-RIGHT: 22.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 7.5pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Courier New"; COLOR: #1f497d'>o</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>30%-40% drop in performance observed<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpFirst style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Carolyn<o:p></o:p></SPAN></P>
<P class=MsoNormalCxSpMiddle style="MARGIN: 0in 0in 0pt; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-TOP: #e1e1e1 1pt solid; BORDER-RIGHT: medium none; BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 3pt; PADDING-LEFT: 0in; BORDER-LEFT: medium none; PADDING-RIGHT: 0in">
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>From:</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'> nvmewin [<A href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</A>] <B>On Behalf Of </B>SUMAN PRAKASH B<BR><B>Sent:</B> Monday, April 04, 2016 6:35 AM<BR><B>To:</B> Thomas Freeman <<A href="mailto:thomas.freeman@hgst.com">thomas.freeman@hgst.com</A>>; <A href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</A><BR><B>Cc:</B> Seokhwan Kim <<A href="mailto:sukka.kim@samsung.com">sukka.kim@samsung.com</A>>; <A href="mailto:anshul@samsung.com">anshul@samsung.com</A>; MANOJ THAPLIYAL <<A href="mailto:m.thapliyal@samsung.com">m.thapliyal@samsung.com</A>>; <A href="mailto:tru.nguyen@ssi.samsung.com">tru.nguyen@ssi.samsung.com</A><BR><B>Subject:</B> Re: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Hi all,</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>I am sending the updated patch incorporating feedback from Tom. The changes are listed below. The password is samsungnvme</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>1. Moved the StorPortFreePool()from IoCompletionRoutine() to NvmeInitCallback() - NvmeWaitOnNamespaceReady.</SPAN></SPAN><o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>2. In NVMeRunningWaitOnNamespaceReady(), the READ command will be sent only when the NS is ATTACHED and the pLunExt->slotStatus is ONLINE. pLunExt->slotStatus willnot be ONLINE if lun extension is zero'ed out.</SPAN></SPAN><o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>3. In NvmeInitCallBack(), in case NVMeWaitOnNamespaceReady, the READ will be retried only if SC = 0x82, else move to the next NS. If the NS LBA format is unsupported, miniport sends the READ command, for which device will return SC=0xb, and miniport will move to next NS.</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><STRONG><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>To Mandatory reviewers</SPAN></STRONG><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>: Can we get feedback or approval for this patch before 7th April.</SPAN></SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'> </SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Thanks,</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Suman</SPAN><SPAN style="COLOR: black"><o:p></o:p></SPAN></P>
<P> <o:p></o:p></P>
<P>------- <B>Original Message</B> -------<o:p></o:p></P>
<P><B>Sender</B> : SUMAN PRAKASH B<<A href="mailto:suman.p@samsung.com">suman.p@samsung.com</A>> Senior Chief Engineer/SSIR-SSD Solutions/Samsung Electronics<o:p></o:p></P>
<P><B>Date</B> : Mar 29, 2016 20:27 (GMT+05:30)<o:p></o:p></P>
<P><B>Title</B> : Re: RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Hi Tom,</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Thanks for the review comments. Please find my replies below:</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>1. nvmeStd.c::IoCompletionRoutine, when checking for NVMeWaitOnNamespaceReady, would it be better to make this check and free the buffer in NvmeInitCallback (when processing NVMeWaitOnNamespaceReady). The check in IoCompletionRoutine is executed during the processing of every IO command, but it will only ever be TRUE during initialization.<BR><STRONG><U><SPAN style='FONT-FAMILY: "Calibri",sans-serif; COLOR: blue'>[Suman]</SPAN></U></STRONG><STRONG><SPAN style='FONT-FAMILY: "Calibri",sans-serif; COLOR: blue'> </SPAN></STRONG><SPAN style="COLOR: blue">Agreed. We will move the StorPortFreePool to NvmeInitCallback.</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>2.a. With your change, in the method NVMeRunningWaitOnNamespaceReady the driver picks the next lun in the list and issues a READ to that namespace. <BR><STRONG><U><SPAN style='FONT-FAMILY: "Calibri",sans-serif; COLOR: blue'>[Suman]</SPAN></U></STRONG><SPAN style="COLOR: blue"> Following changes are made:<BR>a. In NVMeRunningWaitOnNamespaceReady(), the READ command will be sent only when the NS is ATTACHED and the pLunExt->slotStatus is ONLINE. pLunExt->slotStatus will not be ONLINE if lun extension is zero'ed out.<BR>b. In NvmeInitCallBack(), in case NVMeWaitOnNamespaceReady, the READ will be retried only if SC = 0x82, else move to the next NS. If the NS LBA format is unsupported, miniport sends the READ command, for which device will return SC=0xb, and miniport will move to next NS.</SPAN></SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Let me know if you have any questions.</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><BR>We will share the modified code once others share their feedback. Can we get feedback from other companies by 5th April?</SPAN><o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><BR>Thanks,<BR>Suman</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P>------- <B>Original Message</B> -------<o:p></o:p></P>
<P><B>Sender</B> : Thomas Freeman<<A href="mailto:thomas.freeman@hgst.com">thomas.freeman@hgst.com</A>><o:p></o:p></P>
<P><B>Date</B> : Mar 29, 2016 00:48 (GMT+05:30)<o:p></o:p></P>
<P><B>Title</B> : RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></P>
<P> <o:p></o:p></P>
<P class=MsoNormal style="MARGIN-BOTTOM: 0pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 0in"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Hi Suman,<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN-BOTTOM: 0pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 15pt; mso-margin-top-alt: 0in"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>It looks good. <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN-BOTTOM: 0pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 15pt; mso-margin-top-alt: 0in"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>I have a few comments here:<o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 10.5pt; MARGIN-RIGHT: 22.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 5.0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>1.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>nvmeStd.c::IoCompletionRoutine, when checking for NVMeWaitOnNamespaceReady, would it be better to make this check and free the buffer in NvmeInitCallback (when processing NVMeWaitOnNamespaceReady). The check in IoCompletionRoutine is executed during the processing of every IO command, but it will only ever be TRUE during initialization.<o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 10.5pt; MARGIN-RIGHT: 37.5pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 5.0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>2.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>I ran into a few problems, here are the details:<o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 10.5pt; MARGIN-RIGHT: 37.5pt; mso-margin-top-alt: 5.0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>*My device configuration: I'm testing a device that supports NS management and it has multiple namespaces. Some of those namespaces are not attached. The format of some of those namespaces is not supported by the driver (e.g. LBA contains metadata) <o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 46.5pt; MARGIN-RIGHT: 15pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 5.0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>a.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>With your change, in the method NVMeRunningWaitOnNamespaceReady the driver picks the next lun in the list and issues a READ to that namespace. <o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 82.5pt; MARGIN-RIGHT: 15pt; TEXT-INDENT: -9pt; mso-margin-top-alt: 5.0pt"><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>i.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>If that lun is a detached namespace, the READ fails with a status code of 0xb. The driver attempts to retry until the READ is successful, but the command will never succeed. <o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 82.5pt; MARGIN-RIGHT: 15pt; TEXT-INDENT: -9pt; mso-margin-top-alt: 5.0pt"><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>ii.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>During initialization, if the driver detects a namespace that is in an unsupported format, it zero's out that LUN entry, but leaves that zero’ed entry in the LUN extension list. When NVMeRunningWaitOnNamespaceReady is processing the list, it does not recognize this as a zero'ed out entry. Rather is attempts a READ from this namespace (the NSID is 0 since the init code zero'ed out that Lun list entry). The READ and all of its retries fail with a status code of 0xb.<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN-RIGHT: 7.5pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Proposed fix:<o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 0.25in; MARGIN-RIGHT: 30pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 5.0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>1.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Before issuing a READ, ensure that namespace is attached and a valid format. If not, increment the counters and move to the next Lun. <o:p></o:p></SPAN></P>
<P class=MsoListParagraph style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 0.25in; MARGIN-RIGHT: 30pt; TEXT-INDENT: -0.25in; mso-margin-top-alt: 5.0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>2.</SPAN><SPAN style="FONT-SIZE: 7pt; COLOR: #1f497d"> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Also, in NVMeInitCallback, when handling the case NVMeWaitOnNamespaceReady, instead of looking for an SC of 0x00, only issue a retry if the command fails with SC = 0x82 (NS not ready). <o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN-RIGHT: 37.5pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'>Let me know if you have any questions.<o:p></o:p></SPAN></P>
<DIV>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><B><SPAN style='FONT-SIZE: 10pt; FONT-FAMILY: "Arial",sans-serif; COLOR: #1f497d'>Tom Freeman<o:p></o:p></SPAN></B></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><B><SPAN style='FONT-SIZE: 10pt; FONT-FAMILY: "Arial",sans-serif; COLOR: #1f497d'>Software Engineer, Device Manager and Driver Development<o:p></o:p></SPAN></B></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 10pt; FONT-FAMILY: "Arial",sans-serif; COLOR: #1f497d'>Western Digital Corporation<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 10pt; FONT-FAMILY: "Arial",sans-serif; COLOR: #1f497d'>e. <A href="mailto:Thomas.freeman@hgst.com">Thomas.freeman@hgst.com</A><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 10pt; FONT-FAMILY: "Arial",sans-serif; COLOR: #1f497d'>o. +1-507-322-2311<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><IMG id=Picture_x0020_5 border=0 alt=http://www.samsung.net/service/ml/AttachController/image001.jpg?cmd=downdirectly&filepath=/LOCAL/ML/CACHE/s/20160406/image001.jpg@01D18E55.4C7BA130309suman.p&contentType=IMAGE/JPEG;charset=UTF8&msgno=309&partno=2&foldername=INBOX&msguid=48143 src="cid:BGFC2LL5XOK0@namo.co.kr" width=150 height=47><o:p></o:p></SPAN></P></DIV>
<P class=MsoNormal><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #1f497d'><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-TOP: #e1e1e1 1pt solid; BORDER-RIGHT: medium none; BORDER-BOTTOM: medium none; PADDING-BOTTOM: 0in; PADDING-TOP: 3pt; PADDING-LEFT: 0in; BORDER-LEFT: medium none; PADDING-RIGHT: 0in">
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>From:</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'> nvmewin [<A href="mailto:nvmewin-bounces@lists.openfabrics.org">mailto:nvmewin-bounces@lists.openfabrics.org</A>] <B>On Behalf Of </B>SUMAN PRAKASH B<BR><B>Sent:</B> Wednesday, March 23, 2016 7:27 AM<BR><B>To:</B> <A href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</A><BR><B>Cc:</B> Seokhwan Kim <<A href="mailto:sukka.kim@samsung.com">sukka.kim@samsung.com</A>>; MANOJ THAPLIYAL <<A href="mailto:m.thapliyal@samsung.com">m.thapliyal@samsung.com</A>>; <A href="mailto:tru.nguyen@ssi.samsung.com">tru.nguyen@ssi.samsung.com</A><BR><B>Subject:</B> [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11.5pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Hi all,</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11.5pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>This patch includes changes for </SPAN></SPAN><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>optimizing the disk initialization performance and relevant changes.</SPAN></SPAN><o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11.5pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>I have made a detailed overview of the changes in the attached doc file(the contents are also copied here below) and the attached zip file contains the source code.</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11.5pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Password is samsungnvme</SPAN></SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11.5pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Please let me know if you have any questions.<o:p></o:p></SPAN></SPAN></P>
<P> <o:p></o:p></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Thanks,</SPAN><SPAN style="COLOR: black"><o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Suman</SPAN><SPAN style="COLOR: black"><o:p></o:p></SPAN></P>
<P> <o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 11.5pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>******************</SPAN></SPAN><o:p></o:p></P>
<P><SPAN class=apple-style-span><SPAN style='FONT-SIZE: 12pt; FONT-FAMILY: "Verdana",sans-serif; COLOR: black'> <o:p></o:p></SPAN></SPAN></P>
<P><B><U><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Disk Initialization Performance Optimization:</SPAN></U></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><BR>We can use the StorPortInitializePerfOpts(), PERF_CONFIGURATION_DATA.MessageTargets which provides the array of MSI numbers corresponding to each logical processor. This is an alternative of using the Learning cores logic implemented in the OFA driver.<o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'> <o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Also this will directly reduce the time taken for the disk to be enumerated after a device hot insert.<o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>The current OFA driver does the following in its initialization path, let’s say on a server which has 32 logical processors and device which supports 32 queues –<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>1.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Identify controller<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>2.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Identify namespace - for N number of namespaces<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>3.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Set features - Interrupt coalescing, number of queues, lba range type.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>4.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Create IO completion queue - 32 commands<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>5.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Create IO submission queue - 32 commands<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>6.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>LearnMapping - 32 Read commands<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>7.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>ReSetupQueues - 32 Delete Sub queues + 32 Delete completion queues<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>8.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Create IO completion queue - 32 commands<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>9.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Create IO submission queue - 32 commands<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>10.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Complete initialization state machine<o:p></o:p></SPAN></P>
<P> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>As can be observed, during disk initialization, around 224 commands are processed for setting up the IO queues and associate the MSI-x number to each queues. If we use StorPortInitializePerfOpts(), we required only 64 commands instead of 224 commands. On a server which as 120 logical processors, 840 commands are required for setting up the IO queues and associate the MSI-x number to each queues. If learning cores is avoided, only 240 commands are required instead of 840 commands.<o:p></o:p></SPAN></P>
<P> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Also we can fall back to learning cores if the API StorPortInitiailzePerfOpts() fails.<o:p></o:p></SPAN></P>
<P> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>We see improved device up time after this change. Also, if the number of queues supported by the device is less than the number of logical processors, the driver does not execute the learning cores, hence there won’t be any improvement even if we use StorPortInitializePerfOpts().<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 7.5pt"> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<TABLE class=MsoNormalTable style="WIDTH: 371pt; BORDER-COLLAPSE: collapse" cellSpacing=0 cellPadding=0 width=495 border=0>
<TBODY>
<TR style="HEIGHT: 16.6pt">
<TD style="BORDER-TOP: black 1pt solid; HEIGHT: 16.6pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 370.5pt; BACKGROUND: #ffcc99; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: black 1pt solid; PADDING-RIGHT: 0in; border-image: none" vAlign=bottom width=494 colSpan=4>
<P style="TEXT-ALIGN: center" align=center><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: #3f3f76'>Server with 32 logical processors:</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD></TR>
<TR style="HEIGHT: 15pt">
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 62.25pt; BACKGROUND: #a5a5a5; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: black 1pt solid; PADDING-RIGHT: 0in" rowSpan=3 width=83>
<P style="TEXT-ALIGN: center" align=center><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: white'>OFA version</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 306.75pt; BACKGROUND: #a5a5a5; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" width=409 colSpan=3>
<P style="TEXT-ALIGN: center" align=center><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: white'>disk up time in seconds</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD></TR>
<TR style="HEIGHT: 15pt">
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 205.5pt; BACKGROUND: #a5a5a5; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" width=274 colSpan=2>
<P style="TEXT-ALIGN: center" align=center><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: white'>Disk capacity = 400 GB</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 100.5pt; BACKGROUND: #a5a5a5; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" rowSpan=2 width=134>
<P style="TEXT-ALIGN: center" align=center><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: white'>Disk capacity = 1.6 TB</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD></TR>
<TR style="HEIGHT: 15pt">
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 111.75pt; BACKGROUND: #a5a5a5; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" width=149>
<P><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: white'>Disk from vendor 1</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 93pt; BACKGROUND: #a5a5a5; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" width=124>
<P><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: white'>Disk from vendor 2</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD></TR>
<TR style="HEIGHT: 15pt">
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 62.25pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: black 1pt solid; PADDING-RIGHT: 0in" vAlign=bottom width=83>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Rev 133</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 111.75pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" vAlign=bottom width=149>
<P style="TEXT-ALIGN: right" align=right><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>14</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 93pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" vAlign=bottom width=124>
<P style="TEXT-ALIGN: right" align=right><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>6.5</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 15pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 100.5pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" vAlign=bottom width=134>
<P style="TEXT-ALIGN: right" align=right><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>14.5</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD></TR>
<TR style="HEIGHT: 8.05pt">
<TD style="BORDER-TOP: 0px; HEIGHT: 8.05pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 62.25pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: black 1pt solid; PADDING-RIGHT: 0in" vAlign=bottom width=83>
<P><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>Latest</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 8.05pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 111.75pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" vAlign=bottom width=149>
<P style="TEXT-ALIGN: right" align=right><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>5</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 8.05pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 93pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" vAlign=bottom width=124>
<P style="TEXT-ALIGN: right" align=right><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>5</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD>
<TD style="BORDER-TOP: 0px; HEIGHT: 8.05pt; BORDER-RIGHT: windowtext 1pt solid; WIDTH: 100.5pt; BORDER-BOTTOM: black 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: 0px; PADDING-RIGHT: 0in" vAlign=bottom width=134>
<P style="TEXT-ALIGN: right" align=right><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif; COLOR: black'>13.5</SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P></TD></TR></TBODY></TABLE>
<P><B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>PS</SPAN></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>: data may change for different vendor SSDs<o:p></o:p></SPAN></P>
<P> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P><B><U><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Code changes:</SPAN></U></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto">1.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Changes w.r.t StorPortInitializePerfOpts().<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto">a.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>In NVMeInitialize(), moved the StorPortInitializePerfOpts() after NVMeEnumMsiMessages() to set the LastRedirectionMessageNumber in StorPortInitializePerfOpts().<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto">b.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Set the flags STOR_PERF_INTERRUPT_MESSAGE_RANGES and STOR_PERF_ADV_CONFIG_LOCALITY, and values FirstRedirectionMessageNumber, LastRedirectionMessageNumber and MessageTargets in StorPortInitializePerfOpts() to get the MSIx-Core mapping in MessageTargets. If this API returns success, the learning cores can be skipped.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto">c.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>If the StorPortInitializePerfOpts() fails, in NVMeMsiMapCores(), the mapping of msix to cores in assigned sequentially, and learning cores is executed. During learning cores, in IoCompletionRoutine(), the msix to core is re-mapped. If the StorPortInitializePerfOpts() succeeds, in NVMeMsiMapCores(), the mapping of msix to cores is taken from MessageTargets and learnig cores is skipped.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; mso-add-space: auto"> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto">2.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>When the learning cores is skipped, the controller initialization completes faster. But we have observed that in some devices, the Namespace is not ready to process I/O at this point. And when kernel send I/O, the device returns SC = NAMESPACE_NOT_READY and miniport returns SCSI_SENSEQ_BECOMING_READY, for which storport retries after some time. If the device takes too long to initialize the namespace, the storport gives up and shows as Uninitialized in the disk mgmt.<BR>Hence the controller initialization has to be completed after Namespace is ready. For this, a new state is introduce in the NVMeRunning(), which waits till the NS is ready. <o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto"><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>a.</SPAN><SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Introduced a new state NVMeWaitOnNamespaceReady in NVMeRunning().<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto">b.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>In IoCompletionRoutine(), determine which CQ to look in based on WaitOnNamespaceReady state.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto">c.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>In NVMeInitCallback(), implemented call back for NVMeWaitOnNamespaceReady.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 1in; TEXT-INDENT: -0.25in; mso-add-space: auto">d.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>In IoCompletionRoutine(), free the read buffer for namespaceready.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in"><B><U><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Note</SPAN></U></B><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>: <o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in; mso-add-space: auto">a.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>We have observed that higher capacity Namespaces take too long to initialize, hence the passiveTimeout value in NVMePassiveInitialize() is not sufficient. We need to increase the timeout value based on vendor requirements. <o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.75in; TEXT-INDENT: -0.25in; mso-add-space: auto">b.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>b. Checking for Namespace ready is skipped during dump/hibernation mode and resume from hibernation.<o:p></o:p></SPAN></P>
<P> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto">3.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>Usually, the number of MSIx supported by device and MSIx granted(StorPortGetMSIInfo) will be number of IO Queue + 1 Admin Queue. But, we share the Admin queue and first I/O queue in core 0, and hence MSIx 0 is shared between admin queue and first I/O queue. Incase of active cores more than Queues supported, one of the MSGID should not be considered. Made changes in In NVMeEnumMsiMessages() accordingly.<BR>For example, cores = 32, Admin + IO queues = 1 + 8, then MsgID(in NVMeEnumMsiMessages()) should be 8.<o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; mso-add-space: auto"> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P style="MARGIN-LEFT: 0.5in; TEXT-INDENT: -0.25in; mso-add-space: auto">4.<SPAN style='FONT-SIZE: 7pt; FONT-FAMILY: "Times New Roman",serif'> </SPAN><SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'>In IoCompletionRoutine(), for learning cores, only if MSIGranted is less than active cores, the QueueNo will be remapped in sequential manner. Otherwise QueueNo remains same as before.<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 7.5pt"> <SPAN style='FONT-SIZE: 11pt; FONT-FAMILY: "Calibri",sans-serif'><o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="BORDER-TOP: windowtext 1pt solid; BORDER-RIGHT: windowtext 1pt solid; BORDER-BOTTOM: windowtext 1pt solid; PADDING-BOTTOM: 0in; PADDING-TOP: 0in; PADDING-LEFT: 0in; BORDER-LEFT: windowtext 1pt solid; PADDING-RIGHT: 0in"><IMG id=Picture_x0020_1 border=0 alt="Image removed by sender." src="cid:N3WZA6X7LP7K@namo.co.kr" width=100 height=100></SPAN><o:p></o:p></P>
<P class=p1><STRONG><SPAN style='FONT-FAMILY: "Arial",sans-serif'>Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:</SPAN></STRONG><o:p></o:p></P>
<P class=p1><SPAN class=s1>This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.</SPAN><o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> </P>
<P> </P>
<P> </P>
<P> </P>
<TABLE id=confidentialsignimg>
<TBODY>
<TR>
<TD NAMO_LOCK>
<P><IMG border=0 src="cid:4CBBEM6S04A2@namo.co.kr"></P></TD></TR></TBODY></TABLE></DIV></BODY></HTML><img src='http://ext.samsung.net/mailcheck/SeenTimeChecker?do=9de5907ae3594b95bbb95d77db1aaa489b4e2ca6030fddd87d9badbdf7e30042d1afaaba7860cdcd9564217c646641ad61e16949eaa607501b20909a04efd4d2748cfe1d4e847419cf878f9a26ce15a0' border=0 width=0 height=0 style='display:none'>