<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1169368057;
mso-list-type:hybrid;
mso-list-template-ids:-1877205812 -1160607260 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:8;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Calibri","sans-serif";
mso-fareast-font-family:Calibri;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Firstly, thank you, Carolyn, for adding the changes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Please review/test the patch as soon as possible and I will start to collect approvals after July 15.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Thank you all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> nvmewin-bounces@lists.openfabrics.org [mailto:nvmewin-bounces@lists.openfabrics.org]
<b>On Behalf Of </b>Foster, Carolyn D<br>
<b>Sent:</b> Tuesday, July 08, 2014 1:10 PM<br>
<b>To:</b> nvmewin@lists.openfabrics.org<br>
<b>Subject:</b> [nvmewin] NVMe OFA Patch Update for CPU Learning Mode<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<pre>Content-Type: text/plain; charset=UTF-8<o:p></o:p></pre>
<pre>Content-Transfer-Encoding: 8bit<o:p></o:p></pre>
<pre>Date: %%SENT_DATE%%<o:p></o:p></pre>
<pre>Subject: Suspect Message Quarantined <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre><o:p> </o:p></pre>
<pre>WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons:<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>%%DESC%%<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The full message and the attachment have been stored in the quarantine.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>The identifier for this message is '%%QID%%'.<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Access the quarantine at: <o:p></o:p></pre>
<pre><a href="https://puremessage.pmc-sierra.bc.ca:28443/">https://puremessage.pmc-sierra.bc.ca:28443/</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>For more information on PMC's Anti-Spam system:<o:p></o:p></pre>
<pre><a href="http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ">http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>IT Services<o:p></o:p></pre>
<pre>PureMessage Admin <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<p class="MsoNormal">Hi Everyone,<o:p></o:p></p>
<p class="MsoNormal">I have a patch for review with some fixes for the recent CPU patch that I submitted. Please send feedback by Tuesday, July 15<sup>th</sup>. The password is intel123<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Problem statement:<o:p></o:p></p>
<p class="MsoNormal">There were scenarios observed where the CPU core to MSI vector mapping weren’t assigning the IO queue pairs appropriately. We would see that queue pairs weren’t being assigned in a unique manner during the learning mode phase. There were
also cases when an NVMe Reset would clear out the core table mapping that was set up during the initial learning mode. The incorrect mapping resulted in IOs not being completed, which could result in crashes or slow performance.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Changes made:<o:p></o:p></p>
<p class="MsoNormal">The original algorithm relied on the initial queue assignment, which wasn’t guaranteed to be unique. The fix was to use a counter to track the IO queues that had already been mapped to a logical CPU core and MSI vector. Now when a CPU
core is mapped to an MSI vector, we assign the IO queue pair using the current counter value and increment the counter.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Files changed:<o:p></o:p></p>
<p class="MsoNormal">nvmeInit.c<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>In InitSubQueue there was a bug where we would go into shared mode because the number of queues allocated was less than the number of cores, it should have been comparing the number of queues allocated to the maxCore variable.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>In InitCplQueue we come through this function after a reset. Learning mode is already complete, so we don’t need to reset the core table message ID in this case.
<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>In InitCallBack when learning mode is complete, there’s a check to make sure we mapped all the queues to MSI vectors, if we didn’t, then we’ll go back to shared mode.<o:p></o:p></p>
<p class="MsoNormal">nvmeStat.c<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>There were a few places where LearningCores was assigned the wrong value, LearningCores should always be the number of active cores, that’s how it’s clear that learning mode is complete.<o:p></o:p></p>
<p class="MsoNormal">nvmeStd.c<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo2"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>The main change here is in the IoCompletionDpcRoutine, during learning mode. We now track if an MSI vector has been learned, and if it hasn’t then we set up the mapping of CPU core, to MSI vector, to IO queue pair. We also assign the
queue pair using the new counter, NumIoQMapped.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Unit tests<o:p></o:p></p>
<p class="MsoNormal">Win 7, Win8.1, Server 2012 r2, server 2008 r2<o:p></o:p></p>
<p class="MsoNormal">Boot<o:p></o:p></p>
<p class="MsoNormal">Hibernate<o:p></o:p></p>
<p class="MsoNormal">Enable/Disable Controller<o:p></o:p></p>
<p class="MsoNormal">Iometer/SDStress<o:p></o:p></p>
<p class="MsoNormal">SCSI compliance<o:p></o:p></p>
<p class="MsoNormal">Client and Server Platforms<o:p></o:p></p>
<p class="MsoNormal">IO timeouts with resets<o:p></o:p></p>
</div>
</body>
</html>