[nvmewin] NVMe OFA Patch Update for CPU Learning Mode

[Alex Chang]

Hi all,

Firstly, thank you, Carolyn, for adding the changes.
Please review/test the patch as soon as possible and I will start to collect approvals after July 15.

Thank you all,
Alex

From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D
Sent: Tuesday, July 08, 2014 1:10 PM
To: nvmewin at lists.openfabrics.org
Subject: [nvmewin] NVMe OFA Patch Update for CPU Learning Mode


Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

Date: %%SENT_DATE%%

Subject: Suspect Message Quarantined







WARNING: The virus scanner was unable to scan an attachment in an email message sent to you.  This attachment could possibly contain viruses or other malicious programs.  The attachment could not be scanned for the following reasons:



%%DESC%%



The full message and the attachment have been stored in the quarantine.



The identifier for this message is '%%QID%%'.



Access the quarantine at:

https://puremessage.pmc-sierra.bc.ca:28443/



For more information on PMC's Anti-Spam system:

http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ



IT Services

PureMessage Admin


Hi Everyone,
I have a patch for review with some fixes for the recent CPU patch that I submitted.  Please send feedback by Tuesday, July 15th.  The password is intel123

Problem statement:
There were scenarios observed where the CPU core to MSI vector mapping weren't assigning the IO queue pairs appropriately.  We would see that queue pairs weren't being assigned in a unique manner during the learning mode phase.  There were also cases when an NVMe Reset would clear out the core table mapping that was set up during the initial learning mode.  The incorrect mapping resulted in IOs not being completed, which could result in crashes or slow performance.

Changes made:
The original algorithm relied on the initial queue assignment, which wasn't guaranteed to be unique.  The fix was to use a counter to track the IO queues that had already been mapped to a logical CPU core and MSI vector.  Now when a CPU core is mapped to an MSI vector, we assign the IO queue pair using the current counter value and increment the counter.

Files changed:
nvmeInit.c

-          In InitSubQueue there was a bug where we would go into shared mode because the number of queues allocated was less than the number of cores, it should have been comparing the number of queues allocated to the maxCore variable.

-          In InitCplQueue we come through this function after a reset.  Learning mode is already complete, so we don't need to reset the core table message ID in this case.

-          In InitCallBack when learning mode is complete, there's a check to make sure we mapped all the queues to MSI vectors, if we didn't, then we'll go back to shared mode.
nvmeStat.c

-          There were a few places where LearningCores was assigned the wrong value, LearningCores should always be the number of active cores, that's how it's clear that learning mode is complete.
nvmeStd.c

-          The main change here is in the IoCompletionDpcRoutine, during learning mode.  We now track if an MSI vector has been learned, and if it hasn't then we set up the mapping of CPU core, to MSI vector, to IO queue pair.  We also assign the queue pair using the new counter, NumIoQMapped.

Unit tests
Win 7, Win8.1, Server 2012 r2, server 2008 r2
Boot
Hibernate
Enable/Disable Controller
Iometer/SDStress
SCSI compliance
Client and Server Platforms
IO timeouts with resets
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20140708/bf6b5753/attachment.html>