[nvmewin] NVMe OFA patch for CPU Learning mode

[Knoblaugh, Rick]

Hi Carolyn,
                           Thanks for catching that. I should hopefully be done with approval by EOD.

Greatly appreciate it.

               -Rick

From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D
Sent: Wednesday, April 23, 2014 10:47 AM
To: Alex Chang; nvmewin at lists.openfabrics.org
Subject: Re: [nvmewin] NVMe OFA patch for CPU Learning mode

Hello Alex and Rick, We did find an issue with this patch where it was possible for the number of MSI messages granted to be 0.  In this case the number of IO queues would then be set to 0.  I have made a small change to fix this in the attached zip file.

The password is intel123

Thanks,
Carolyn

From: Alex Chang [mailto:Alex.Chang at pmcs.com]
Sent: Tuesday, April 22, 2014 7:53 PM
To: Foster, Carolyn D; nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: RE: NVMe OFA patch for CPU Learning mode

Hi Rick,

I had finished reviewing/testing on the patch. When you've done so, please let us know with your approval.

Thanks,
Alex

From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com]
Sent: Wednesday, April 16, 2014 3:00 PM
To: Alex Chang; nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: RE: NVMe OFA patch for CPU Learning mode


Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

Date: %%SENT_DATE%%

Subject: Suspect Message Quarantined







WARNING: The virus scanner was unable to scan an attachment in an email message sent to you.  This attachment could possibly contain viruses or other malicious programs.  The attachment could not be scanned for the following reasons:



%%DESC%%



The full message and the attachment have been stored in the quarantine.



The identifier for this message is '%%QID%%'.



Access the quarantine at:

https://puremessage.pmc-sierra.bc.ca:28443/



For more information on PMC's Anti-Spam system:

http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ



IT Services

PureMessage Admin


My apologies, I sent out the wrong version, thank you Alex.  I have attached the correct rebased version.

The password is still intel123

Carolyn

From: Alex Chang [mailto:Alex.Chang at pmcs.com]
Sent: Wednesday, April 16, 2014 11:31 AM
To: Foster, Carolyn D; nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: RE: NVMe OFA patch for CPU Learning mode

Hi Carolyn,

Did you re-base the sources before adding your changes? The patch you sent out seems not including what I added in Patch#24.

Thanks,
Alex

From: nvmewin-bounces at lists.openfabrics.org<mailto:nvmewin-bounces at lists.openfabrics.org> [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D
Sent: Tuesday, April 15, 2014 4:07 PM
To: nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: [nvmewin] NVMe OFA patch for CPU Learning mode


Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

Date: %%SENT_DATE%%

Subject: Suspect Message Quarantined







WARNING: The virus scanner was unable to scan an attachment in an email message sent to you.  This attachment could possibly contain viruses or other malicious programs.  The attachment could not be scanned for the following reasons:



%%DESC%%



The full message and the attachment have been stored in the quarantine.



The identifier for this message is '%%QID%%'.



Access the quarantine at:

https://puremessage.pmc-sierra.bc.ca:28443/



For more information on PMC's Anti-Spam system:

http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ



IT Services

PureMessage Admin


The password is intel1234

Problem statement:
The current OFA driver assumes a one to one mapping of MSI vectors, queues and CPU cores.  If there is not a one to one mapping then the driver does not go through learning mode and we see a performance drop.  Learning mode is how we maintain our source core interrupt steering, where we map MSI vectors to CPU cores.  This mapping allows the driver to issue and complete commands on the same CPU core.

Proposed changes:
Instead of giving up on learn mapping if there are differences between MSI vectors and CPU cores, we will proceed as normal with learning mode.  We allocate the Core table for the max number of cores, and if at the end of learning mode, any CPU cores have not been mapped, we will map them to Submission queues in a round robin fashion.  We also take into account if the MSI vectors are not mapped contiguously, or if the number of submission and completion queues are different from each other and the number of cores.  These changes still won't have 100% functionality of the source core interrupt steering, but performance is better than if we don't try at all.  Most of the changes are in the initialization path, there was no change to the IO path.

Also in this patch is the removal of the #defines for the CHATHAM prototype hardware.

Unit Tests:
Tested the following on Windows 7 and Windows 8 based systems.
Booted from a system with more CPU cores than MSI vectors.
IO stress on a setup with fewer IO queues than CPU cores and MSI vectors
Ran SCSI compliance tests
Ran SDStress
Ran IOmeter
Hibernate
Format (quick and slow) of MBR and GPT
Install/Uninstall, Enable/Disable


Thanks!
Carolyn

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20140423/2cc40a76/attachment.html>