<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<title>Samsung Enterprise Portal mySingle</title>
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-believe-normal-left:yes;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin-top:3.75pt;
margin-right:0in;
margin-bottom:3.75pt;
margin-left:0in;
font-size:9.0pt;
font-family:"Arial",sans-serif;}
p.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing
{mso-style-priority:1;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:.5in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.p1, li.p1, div.p1
{mso-style-name:p1;
mso-style-priority:99;
margin-top:3.75pt;
margin-right:0in;
margin-bottom:3.75pt;
margin-left:0in;
font-size:9.0pt;
font-family:"Arial",sans-serif;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;}
span.EmailStyle23
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;}
span.s1
{mso-style-name:s1;}
span.EmailStyle25
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:55857837;
mso-list-type:hybrid;
mso-list-template-ids:1069176448 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:376777175;
mso-list-type:hybrid;
mso-list-template-ids:-758583154 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.25in;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:1.25in;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:1.75in;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:2.25in;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:2.75in;
text-indent:-.25in;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:3.25in;
text-indent:-.25in;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:3.75in;
text-indent:-.25in;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:4.25in;
text-indent:-.25in;
font-family:Wingdings;}
@list l2
{mso-list-id:533227866;
mso-list-type:hybrid;
mso-list-template-ids:-1196136308 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l3
{mso-list-id:1778285189;
mso-list-type:hybrid;
mso-list-template-ids:145102556 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l3:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.25in;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:.75in;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:1.25in;
text-indent:-.25in;
font-family:Wingdings;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:1.75in;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:2.25in;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:2.75in;
text-indent:-.25in;
font-family:Wingdings;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:3.25in;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:3.75in;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
margin-left:4.25in;
text-indent:-.25in;
font-family:Wingdings;}
@list l4
{mso-list-id:1820028171;
mso-list-type:hybrid;
mso-list-template-ids:880981156 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l4:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l4:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l4:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l4:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l4:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l4:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l4:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l4:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l4:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l5
{mso-list-id:1967929396;
mso-list-type:hybrid;
mso-list-template-ids:-1501651152 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l5:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l5:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l5:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l5:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l5:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l5:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l5:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l5:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l5:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
--></style><![if mso 9]><style>p.MsoNormal
{margin-left:7.5pt;}
</style><![endif]><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="margin-left:7.5pt;margin-top:7.5pt;margin-right:7.5pt;margin-bottom:7.5pt">
<div class="WordSection1">
<p class="MsoNormalCxSpFirst" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Suman,<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormalCxSpLast" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I have a few comments and suggestions:<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpFirst" style="mso-margin-top-alt:7.5pt;margin-right:7.5pt;margin-bottom:7.5pt;margin-left:.5in;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo2">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Observed performance degradation, potentially related to line 594 in nvmeInit.c – We noticed on some systems we saw a degradation in performance and
we suspect it’s related to this change. If we don’t share MSIX vector 0 between the admin queue and an IO queue we are creating one fewer queue to submit IO to. Did you execute any performance testing before and after these changes? I have included some
details about the system configurations we tested and the observed results below.<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo2">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">nvmeStat.c @ line 784 : if there is zero namespace it is not necessary to go to NVMeWaitOnNamespaceReady , instead we can directly start NVMeStartComplete.<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpLast" style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:0in;margin-left:.5in;margin-bottom:.0001pt;mso-add-space:auto;text-indent:-.25in;mso-list:l0 level1 lfo2">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">nvmeStat.c @line 899 : In crash/Hibernate mode it is not necessary to go to the NVMeWaitOnNamespaceReady.<o:p></o:p></span></p>
<p class="MsoNormalCxSpFirst" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></a></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Performance configuration and data:<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">OS: Windows 8.1 x64<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Workload: 100% sequential Read<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Compared the OFA trunk to the Samsung patch<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="mso-margin-top-alt:7.5pt;margin-right:15.0pt;margin-bottom:7.5pt;margin-left:0in;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormalCxSpLast" style="mso-margin-top-alt:7.5pt;margin-right:15.0pt;margin-bottom:7.5pt;margin-left:0in;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Summary of observed results:<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpFirst" style="mso-margin-top-alt:7.5pt;margin-right:45.0pt;margin-bottom:7.5pt;margin-left:.25in;mso-add-space:auto;text-indent:-.25in;mso-list:l3 level1 lfo7">
<![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">System with fewer than 32 logical CPU cores:
<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-margin-top-alt:7.5pt;margin-right:52.5pt;margin-bottom:7.5pt;margin-left:.75in;mso-add-space:auto;text-indent:-.25in;mso-list:l3 level2 lfo7">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">No delta in performance observed<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-margin-top-alt:7.5pt;margin-right:52.5pt;margin-bottom:7.5pt;margin-left:.25in;mso-add-space:auto;text-indent:-.25in;mso-list:l3 level1 lfo7">
<![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">System with between 32 and 64 cores:
<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-margin-top-alt:7.5pt;margin-right:22.5pt;margin-bottom:7.5pt;margin-left:.75in;mso-add-space:auto;text-indent:-.25in;mso-list:l3 level2 lfo7">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">20%-50% drop in performance observed<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpMiddle" style="mso-margin-top-alt:7.5pt;margin-right:22.5pt;margin-bottom:7.5pt;margin-left:.25in;mso-add-space:auto;text-indent:-.25in;mso-list:l3 level1 lfo7">
<![if !supportLists]><span style="font-size:11.0pt;font-family:Symbol;color:#1F497D"><span style="mso-list:Ignore">·<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">System with more than 64 cores:<o:p></o:p></span></p>
<p class="MsoListParagraphCxSpLast" style="mso-margin-top-alt:7.5pt;margin-right:22.5pt;margin-bottom:7.5pt;margin-left:.75in;mso-add-space:auto;text-indent:-.25in;mso-list:l3 level2 lfo7">
<![if !supportLists]><span style="font-size:11.0pt;font-family:"Courier New";color:#1F497D"><span style="mso-list:Ignore">o<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">30%-40% drop in performance observed<o:p></o:p></span></p>
<p class="MsoNormalCxSpFirst" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Carolyn<o:p></o:p></span></p>
<p class="MsoNormalCxSpMiddle" style="margin:0in;margin-bottom:.0001pt;mso-add-space:auto">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> nvmewin [mailto:nvmewin-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>SUMAN PRAKASH B<br>
<b>Sent:</b> Monday, April 04, 2016 6:35 AM<br>
<b>To:</b> Thomas Freeman <thomas.freeman@hgst.com>; nvmewin@lists.openfabrics.org<br>
<b>Cc:</b> Seokhwan Kim <sukka.kim@samsung.com>; anshul@samsung.com; MANOJ THAPLIYAL <m.thapliyal@samsung.com>; tru.nguyen@ssi.samsung.com<br>
<b>Subject:</b> Re: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Hi all,</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">I am sending the updated patch incorporating feedback from Tom. The changes are listed below. The password is samsungnvme</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">1. Moved the StorPortFreePool()from IoCompletionRoutine() to NvmeInitCallback() - NvmeWaitOnNamespaceReady.</span></span><o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">2. In NVMeRunningWaitOnNamespaceReady(), the READ command will be sent only when the NS is ATTACHED and the pLunExt->slotStatus is ONLINE. pLunExt->slotStatus
willnot be ONLINE if lun extension is zero'ed out.</span></span><o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">3. In NvmeInitCallBack(), in case NVMeWaitOnNamespaceReady, the READ will be retried only if SC = 0x82, else move to the next NS. If the NS LBA format
is unsupported, miniport sends the READ command, for which device will return SC=0xb, and miniport will move to next NS.</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><strong><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">To Mandatory reviewers</span></strong><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">: Can we get feedback or
approval for this patch before 7th April.</span></span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black"> </span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Thanks,</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Suman</span><span style="color:black"><o:p></o:p></span></p>
<p> <o:p></o:p></p>
<p>------- <b>Original Message</b> -------<o:p></o:p></p>
<p><b>Sender</b> : SUMAN PRAKASH B<suman.p@samsung.com> Senior Chief Engineer/SSIR-SSD Solutions/Samsung Electronics<o:p></o:p></p>
<p><b>Date</b> : Mar 29, 2016 20:27 (GMT+05:30)<o:p></o:p></p>
<p><b>Title</b> : Re: RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Hi Tom,</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Thanks for the review comments. Please find my replies below:</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">1. nvmeStd.c::IoCompletionRoutine, when checking for NVMeWaitOnNamespaceReady, would it be better to make this check and free the buffer in NvmeInitCallback (when processing NVMeWaitOnNamespaceReady).
The check in IoCompletionRoutine is executed during the processing of every IO command, but it will only ever be TRUE during initialization.<br>
<strong><u><span style="font-family:"Calibri",sans-serif;color:blue">[Suman]</span></u></strong><strong><span style="font-family:"Calibri",sans-serif;color:blue">
</span></strong><span style="color:blue">Agreed. We will move the StorPortFreePool to NvmeInitCallback.</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">2.a. With your change, in the method NVMeRunningWaitOnNamespaceReady the driver picks the next lun in the list and issues a READ to that namespace.
<br>
<strong><u><span style="font-family:"Calibri",sans-serif;color:blue">[Suman]</span></u></strong><span style="color:blue"> Following changes are made:<br>
a. In NVMeRunningWaitOnNamespaceReady(), the READ command will be sent only when the NS is ATTACHED and the pLunExt->slotStatus is ONLINE. pLunExt->slotStatus will not be ONLINE if lun extension is zero'ed out.<br>
b. In NvmeInitCallBack(), in case NVMeWaitOnNamespaceReady, the READ will be retried only if SC = 0x82, else move to the next NS. If the NS LBA format is unsupported, miniport sends the READ command, for which device will return SC=0xb, and miniport will move
to next NS.</span></span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Let me know if you have any questions.</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><br>
We will share the modified code once others share their feedback. Can we get feedback from other companies by 5th April?</span><o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><br>
Thanks,<br>
Suman</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p>------- <b>Original Message</b> -------<o:p></o:p></p>
<p><b>Sender</b> : Thomas Freeman<thomas.freeman@hgst.com><o:p></o:p></p>
<p><b>Date</b> : Mar 29, 2016 00:48 (GMT+05:30)<o:p></o:p></p>
<p><b>Title</b> : RE: [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></p>
<p> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:7.5pt;margin-bottom:0in;margin-left:0in;margin-bottom:.0001pt">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Suman,<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:15.0pt;margin-bottom:0in;margin-left:0in;margin-bottom:.0001pt">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">It looks good.
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:0in;margin-right:15.0pt;margin-bottom:0in;margin-left:0in;margin-bottom:.0001pt">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I have a few comments here:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:22.5pt;margin-bottom:5.0pt;margin-left:10.5pt;text-indent:-.25in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">1.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">nvmeStd.c::IoCompletionRoutine, when checking for NVMeWaitOnNamespaceReady, would it be better to make this check and free the buffer in NvmeInitCallback (when processing NVMeWaitOnNamespaceReady).
The check in IoCompletionRoutine is executed during the processing of every IO command, but it will only ever be TRUE during initialization.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:37.5pt;margin-bottom:5.0pt;margin-left:10.5pt;text-indent:-.25in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">2.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I ran into a few problems, here are the details:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:37.5pt;margin-bottom:5.0pt;margin-left:10.5pt">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">*My device configuration: I'm testing a device that supports NS management and it has multiple namespaces. Some of those namespaces are not attached. The format of some of those namespaces
is not supported by the driver (e.g. LBA contains metadata) <o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:15.0pt;margin-bottom:5.0pt;margin-left:46.5pt;text-indent:-.25in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">a.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">With your change, in the method NVMeRunningWaitOnNamespaceReady the driver picks the next lun in the list and issues a READ to that namespace.
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:15.0pt;margin-bottom:5.0pt;margin-left:82.5pt;text-indent:-9.0pt">
<span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">i.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">If that lun is a detached namespace, the READ fails with a status code of 0xb. The driver attempts to retry until the READ is successful, but the command will never succeed.
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:15.0pt;margin-bottom:5.0pt;margin-left:82.5pt;text-indent:-9.0pt">
<span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">ii.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">During initialization, if the driver detects a namespace that is in an unsupported format, it zero's out that LUN entry, but leaves that zero’ed entry in the LUN extension
list. When NVMeRunningWaitOnNamespaceReady is processing the list, it does not recognize this as a zero'ed out entry. Rather is attempts a READ from this namespace (the NSID is 0 since the init code zero'ed out that Lun list entry). The READ and all of its
retries fail with a status code of 0xb.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-right:7.5pt"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Proposed fix:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:30.0pt;margin-bottom:5.0pt;margin-left:.25in;text-indent:-.25in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">1.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Before issuing a READ, ensure that namespace is attached and a valid format. If not, increment the counters and move to the next Lun.
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="mso-margin-top-alt:5.0pt;margin-right:30.0pt;margin-bottom:5.0pt;margin-left:.25in;text-indent:-.25in">
<span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">2.</span><span style="font-size:7.0pt;color:#1F497D">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Also, in NVMeInitCallback, when handling the case NVMeWaitOnNamespaceReady, instead of looking for an SC of 0x00, only issue a retry if the command fails with SC = 0x82 (NS
not ready). <o:p></o:p></span></p>
<p class="MsoNormal" style="margin-right:37.5pt"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Let me know if you have any questions.<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">Tom Freeman<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">Software Engineer, Device Manager and Driver Development<o:p></o:p></span></b></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">Western Digital Corporation<o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">e. Thomas.freeman@hgst.com<o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#1F497D">o. +1-507-322-2311<o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><img width="150" height="47" id="Picture_x0020_5" src="cid:image001.jpg@01D18E55.4C7BA130" alt="cid:image001.jpg@01D18E55.4C7BA130"></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> nvmewin [mailto:nvmewin-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>SUMAN PRAKASH B<br>
<b>Sent:</b> Wednesday, March 23, 2016 7:27 AM<br>
<b>To:</b> nvmewin@lists.openfabrics.org<br>
<b>Cc:</b> Seokhwan Kim <sukka.kim@samsung.com>; MANOJ THAPLIYAL <m.thapliyal@samsung.com>; tru.nguyen@ssi.samsung.com<br>
<b>Subject:</b> [nvmewin] Patch with changes for Optimizing disk initialization performance<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.5pt;font-family:"Calibri",sans-serif;color:black">Hi all,</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.5pt;font-family:"Calibri",sans-serif;color:black">This patch includes changes for
</span></span><span class="apple-style-span"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">optimizing the disk initialization performance and relevant changes.</span></span><o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.5pt;font-family:"Calibri",sans-serif;color:black">I have made a detailed overview of the changes in the attached doc file(the contents are also copied here below) and the attached zip file contains
the source code.</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.5pt;font-family:"Calibri",sans-serif;color:black">Password is samsungnvme</span></span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.5pt;font-family:"Calibri",sans-serif;color:black">Please let me know if you have any questions.<o:p></o:p></span></span></p>
<p> <o:p></o:p></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Thanks,</span><span style="color:black"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Suman</span><span style="color:black"><o:p></o:p></span></p>
<p> <o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:11.5pt;font-family:"Calibri",sans-serif;color:black">******************</span></span><o:p></o:p></p>
<p><span class="apple-style-span"><span style="font-size:12.0pt;font-family:"Verdana",sans-serif;color:black"> <o:p></o:p></span></span></p>
<p><b><u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Disk Initialization Performance Optimization:</span></u></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><br>
We can use the StorPortInitializePerfOpts(), PERF_CONFIGURATION_DATA.MessageTargets which provides the array of MSI numbers corresponding to each logical processor. This is an alternative of using the Learning cores logic implemented in the OFA driver.<o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> <o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Also this will directly reduce the time taken for the disk to be enumerated after a device hot insert.<o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">The current OFA driver does the following in its initialization path, let’s say on a server which has 32 logical processors and device which supports 32 queues –<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">1.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Identify controller<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">2.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Identify namespace - for N number of namespaces<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">3.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Set features - Interrupt coalescing, number of queues, lba range type.<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">4.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Create IO completion queue - 32 commands<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">5.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Create IO submission queue - 32 commands<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">6.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">LearnMapping - 32 Read commands<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">7.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">ReSetupQueues - 32 Delete Sub queues + 32 Delete completion queues<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">8.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Create IO completion queue - 32 commands<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">9.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Create IO submission queue - 32 commands<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">10.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Complete initialization state machine<o:p></o:p></span></p>
<p> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">As can be observed, during disk initialization, around 224 commands are processed for setting up the IO queues and associate the MSI-x number to each queues. If we use StorPortInitializePerfOpts(),
we required only 64 commands instead of 224 commands. On a server which as 120 logical processors, 840 commands are required for setting up the IO queues and associate the MSI-x number to each queues. If learning cores is avoided, only 240 commands are required
instead of 840 commands.<o:p></o:p></span></p>
<p> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Also we can fall back to learning cores if the API StorPortInitiailzePerfOpts() fails.<o:p></o:p></span></p>
<p> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">We see improved device up time after this change. Also, if the number of queues supported by the device is less than the number of logical processors, the driver does not execute the learning
cores, hence there won’t be any improvement even if we use StorPortInitializePerfOpts().<o:p></o:p></span></p>
<p style="margin-bottom:7.5pt"> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="495" style="width:371.0pt;border-collapse:collapse">
<tbody>
<tr style="height:16.6pt">
<td width="494" colspan="4" valign="bottom" style="width:370.5pt;border:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#FFCC99;padding:0in 0in 0in 0in;height:16.6pt;border-image: none">
<p align="center" style="text-align:center"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#3F3F76">Server with 32 logical processors:</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
</tr>
<tr style="height:15.0pt">
<td width="83" rowspan="3" style="width:62.25pt;border-top:none;border-left:solid black 1.0pt;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#A5A5A5;padding:0in 0in 0in 0in;height:15.0pt">
<p align="center" style="text-align:center"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:white">OFA version</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="409" colspan="3" style="width:306.75pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#A5A5A5;padding:0in 0in 0in 0in;height:15.0pt">
<p align="center" style="text-align:center"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:white">disk up time in seconds</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
</tr>
<tr style="height:15.0pt">
<td width="274" colspan="2" style="width:205.5pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#A5A5A5;padding:0in 0in 0in 0in;height:15.0pt">
<p align="center" style="text-align:center"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:white">Disk capacity = 400 GB</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="134" rowspan="2" style="width:100.5pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#A5A5A5;padding:0in 0in 0in 0in;height:15.0pt">
<p align="center" style="text-align:center"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:white">Disk capacity = 1.6 TB</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
</tr>
<tr style="height:15.0pt">
<td width="149" style="width:111.75pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#A5A5A5;padding:0in 0in 0in 0in;height:15.0pt">
<p><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:white">Disk from vendor 1</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="124" style="width:93.0pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;background:#A5A5A5;padding:0in 0in 0in 0in;height:15.0pt">
<p><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:white">Disk from vendor 2</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
</tr>
<tr style="height:15.0pt">
<td width="83" valign="bottom" style="width:62.25pt;border-top:none;border-left:solid black 1.0pt;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:15.0pt">
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Rev 133</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="149" valign="bottom" style="width:111.75pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:15.0pt">
<p align="right" style="text-align:right"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">14</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="124" valign="bottom" style="width:93.0pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:15.0pt">
<p align="right" style="text-align:right"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">6.5</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="134" valign="bottom" style="width:100.5pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:15.0pt">
<p align="right" style="text-align:right"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">14.5</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
</tr>
<tr style="height:8.05pt">
<td width="83" valign="bottom" style="width:62.25pt;border-top:none;border-left:solid black 1.0pt;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:8.05pt">
<p><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">Latest</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="149" valign="bottom" style="width:111.75pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:8.05pt">
<p align="right" style="text-align:right"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">5</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="124" valign="bottom" style="width:93.0pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:8.05pt">
<p align="right" style="text-align:right"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">5</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
<td width="134" valign="bottom" style="width:100.5pt;border-top:none;border-left:none;border-bottom:solid black 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 0in 0in 0in;height:8.05pt">
<p align="right" style="text-align:right"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:black">13.5</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
<p><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">PS</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">: data may change for different vendor SSDs<o:p></o:p></span></p>
<p> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p><b><u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Code changes:</span></u></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in">1.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Changes w.r.t StorPortInitializePerfOpts().<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in">a.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">In NVMeInitialize(), moved the StorPortInitializePerfOpts() after NVMeEnumMsiMessages() to set the LastRedirectionMessageNumber in StorPortInitializePerfOpts().<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in">b.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Set the flags STOR_PERF_INTERRUPT_MESSAGE_RANGES and STOR_PERF_ADV_CONFIG_LOCALITY, and values FirstRedirectionMessageNumber, LastRedirectionMessageNumber and MessageTargets in StorPortInitializePerfOpts()
to get the MSIx-Core mapping in MessageTargets. If this API returns success, the learning cores can be skipped.<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in">c.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">If the StorPortInitializePerfOpts() fails, in NVMeMsiMapCores(), the mapping of msix to cores in assigned sequentially, and learning cores is executed. During learning cores, in IoCompletionRoutine(),
the msix to core is re-mapped. If the StorPortInitializePerfOpts() succeeds, in NVMeMsiMapCores(), the mapping of msix to cores is taken from MessageTargets and learnig cores is skipped.<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto"> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in">2.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">When the learning cores is skipped, the controller initialization completes faster. But we have observed that in some devices, the Namespace is not ready to process I/O at this point. And
when kernel send I/O, the device returns SC = NAMESPACE_NOT_READY and miniport returns SCSI_SENSEQ_BECOMING_READY, for which storport retries after some time. If the device takes too long to initialize the namespace, the storport gives up and shows as Uninitialized
in the disk mgmt.<br>
Hence the controller initialization has to be completed after Namespace is ready. For this, a new state is introduce in the NVMeRunning(), which waits till the NS is ready. <o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">a.</span><span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Introduced a new state NVMeWaitOnNamespaceReady in NVMeRunning().<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in">b.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">In IoCompletionRoutine(), determine which CQ to look in based on WaitOnNamespaceReady state.<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in">c.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">In NVMeInitCallback(), implemented call back for NVMeWaitOnNamespaceReady.<o:p></o:p></span></p>
<p style="margin-left:1.0in;mso-add-space:auto;text-indent:-.25in">d.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">In IoCompletionRoutine(), free the read buffer for namespaceready.<o:p></o:p></span></p>
<p style="margin-left:.5in"><b><u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Note</span></u></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">:
<o:p></o:p></span></p>
<p style="margin-left:.75in;mso-add-space:auto;text-indent:-.25in">a.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">We have observed that higher capacity Namespaces take too long to initialize, hence the passiveTimeout value in NVMePassiveInitialize() is not sufficient. We need to increase the timeout
value based on vendor requirements. <o:p></o:p></span></p>
<p style="margin-left:.75in;mso-add-space:auto;text-indent:-.25in">b.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">b. Checking for Namespace ready is skipped during dump/hibernation mode and resume from hibernation.<o:p></o:p></span></p>
<p> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in">3.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Usually, the number of MSIx supported by device and MSIx granted(StorPortGetMSIInfo) will be number of IO Queue + 1 Admin Queue. But, we share the Admin queue and first I/O queue in core
0, and hence MSIx 0 is shared between admin queue and first I/O queue. Incase of active cores more than Queues supported, one of the MSGID should not be considered. Made changes in In NVMeEnumMsiMessages() accordingly.<br>
For example, cores = 32, Admin + IO queues = 1 + 8, then MsgID(in NVMeEnumMsiMessages()) should be 8.<o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto"> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p style="margin-left:.5in;mso-add-space:auto;text-indent:-.25in">4.<span style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">In IoCompletionRoutine(), for learning cores, only if MSIGranted is less than active cores, the QueueNo will be remapped in sequential manner. Otherwise QueueNo remains same as before.<o:p></o:p></span></p>
<p style="margin-bottom:7.5pt"> <span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="border:solid windowtext 1.0pt;padding:0in"><img width="100" height="100" id="Picture_x0020_1" src="cid:~WRD000.jpg" alt="Image removed by sender."></span><o:p></o:p></p>
<p class="p1"><strong><span style="font-family:"Arial",sans-serif">Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:</span></strong><o:p></o:p></p>
<p class="p1"><span class="s1">This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail
in its entirety from your system.</span><o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<p> <o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellpadding="0" id="confidentialsignimg">
<tbody>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p><img width="520" height="144" id="Picture_x0020_3" src="cid:image002.gif@01D18E55.4C7BA130" alt="cid:image002.gif@01D18E55.4C7BA130"><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin:0in;margin-bottom:.0001pt"><span style="border:solid windowtext 1.0pt;padding:0in"><img width="100" height="100" id="Picture_x0020_4" src="cid:~WRD000.jpg" alt="Image removed by sender."></span><o:p></o:p></p>
</div>
</body>
</html>