From Alex.Chang at pmcs.com Wed Nov 6 11:52:20 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 6 Nov 2013 19:52:20 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> Message-ID: Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dharani.Kotte at sandisk.com Wed Nov 6 11:56:51 2013 From: Dharani.Kotte at sandisk.com (Dharani Kotte) Date: Wed, 6 Nov 2013 19:56:51 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> Message-ID: <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Wed Nov 6 12:00:36 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 6 Nov 2013 20:00:36 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> Message-ID: Great. Thanks. I will do more testing while waiting the approvals from LSI and Intel. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, November 06, 2013 11:57 AM To: Alex Chang; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dharani.Kotte at sandisk.com Wed Nov 6 12:01:28 2013 From: Dharani.Kotte at sandisk.com (Dharani Kotte) Date: Wed, 6 Nov 2013 20:01:28 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> Message-ID: <23EC73C80FB59046A6B7B8EB7B3826593BDAC1B8@SACMBXIP01.sdcorp.global.sandisk.com> Sure, Thank you for your time. Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 12:01 PM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Great. Thanks. I will do more testing while waiting the approvals from LSI and Intel. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, November 06, 2013 11:57 AM To: Alex Chang; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Rick.Knoblaugh at lsi.com Wed Nov 6 13:20:01 2013 From: Rick.Knoblaugh at lsi.com (Knoblaugh, Rick) Date: Wed, 6 Nov 2013 21:20:01 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> Message-ID: <1fcd76855c0847dcb32cbdf7a4ac82cc@DM2PR07MB285.namprd07.prod.outlook.com> Hi Alex, We are good with this change. Thanks, -Rick From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 12:01 PM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Great. Thanks. I will do more testing while waiting the approvals from LSI and Intel. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, November 06, 2013 11:57 AM To: Alex Chang; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Wed Nov 6 13:27:24 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 6 Nov 2013 21:27:24 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: <1fcd76855c0847dcb32cbdf7a4ac82cc@DM2PR07MB285.namprd07.prod.outlook.com> References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> <1fcd76855c0847dcb32cbdf7a4ac82cc@DM2PR07MB285.namprd07.prod.outlook.com> Message-ID: Thanks a lot, Rick. Regards, Alex From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Wednesday, November 06, 2013 1:20 PM To: Alex Chang; Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, We are good with this change. Thanks, -Rick From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 12:01 PM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Great. Thanks. I will do more testing while waiting the approvals from LSI and Intel. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, November 06, 2013 11:57 AM To: Alex Chang; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From rrandall at micron.com Thu Nov 7 13:17:36 2013 From: rrandall at micron.com (Robert Randall (rrandall)) Date: Thu, 7 Nov 2013 21:17:36 +0000 Subject: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges Message-ID: <70C73440F9F7C24F81A11355C292A9B5C10FF73C@NTXBOIMBX04.micron.com> Hello all, We have been testing the NVMe in-box driver included in Windows 8.1 and Server 2012 R2. It does not appear to have any IOCTL support for ADMIN commands. This is very sad. This renders the in-box driver useless for uses cases where VU ADMIN commands are used to, for example, manage NVM Express Namespaces. Have others had a similar experience with the in-box driver? Microsoft appears willing to listen to requests to add capabilities to the in-box driver. This represents an opportunity for all of us participating in the NVM Express ecosystem to express our requirements to Microsoft with one voice. If you believe that the in-box driver is lacking critical features (as I do) I would be happy to coordinate a unified response to the Microsoft program manager responsible for the in-box device driver. Perhaps this would be a useful topic for the next technical conference call? Best regards, Robert Robert Randall mobile: 612.770.9612 rrandall at micron.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From raymond.c.robles at intel.com Thu Nov 7 13:29:10 2013 From: raymond.c.robles at intel.com (Robles, Raymond C) Date: Thu, 7 Nov 2013 21:29:10 +0000 Subject: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges In-Reply-To: <70C73440F9F7C24F81A11355C292A9B5C10FF73C@NTXBOIMBX04.micron.com> References: <70C73440F9F7C24F81A11355C292A9B5C10FF73C@NTXBOIMBX04.micron.com> Message-ID: <49158E750348AA499168FD41D8898360625F3BE9@FMSMSX105.amr.corp.intel.com> Hi Robert, Microsoft has been very adamant about not supporting pass through IOCTLs (via IOCTL_SCSI_MINIPORT) and the potential security threat this mechanism poses. There is a potential for them to listen to suggestions for workarounds, or alternatives, to pass through IOCTLs. However, they have been very clear about not supporting pass through IOCTLs. Thanks, Ray From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Robert Randall (rrandall) Sent: Thursday, November 07, 2013 2:18 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges Hello all, We have been testing the NVMe in-box driver included in Windows 8.1 and Server 2012 R2. It does not appear to have any IOCTL support for ADMIN commands. This is very sad. This renders the in-box driver useless for uses cases where VU ADMIN commands are used to, for example, manage NVM Express Namespaces. Have others had a similar experience with the in-box driver? Microsoft appears willing to listen to requests to add capabilities to the in-box driver. This represents an opportunity for all of us participating in the NVM Express ecosystem to express our requirements to Microsoft with one voice. If you believe that the in-box driver is lacking critical features (as I do) I would be happy to coordinate a unified response to the Microsoft program manager responsible for the in-box device driver. Perhaps this would be a useful topic for the next technical conference call? Best regards, Robert Robert Randall mobile: 612.770.9612 rrandall at micron.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rrandall at micron.com Fri Nov 8 06:20:26 2013 From: rrandall at micron.com (Robert Randall (rrandall)) Date: Fri, 8 Nov 2013 14:20:26 +0000 Subject: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges In-Reply-To: <49158E750348AA499168FD41D8898360625F3BE9@FMSMSX105.amr.corp.intel.com> References: <70C73440F9F7C24F81A11355C292A9B5C10FF73C@NTXBOIMBX04.micron.com> <49158E750348AA499168FD41D8898360625F3BE9@FMSMSX105.amr.corp.intel.com> Message-ID: <70C73440F9F7C24F81A11355C292A9B5C10FFCBF@NTXBOIMBX04.micron.com> Hi Ray, Well stated. However, I confess that I find it odd that Microsoft would not consider providing a WMI interface. I believe that a WMI interface would satisfy their security concerns. Has this been discussed with Microsoft? I would be happy to provide a simple definition of a WMI class which would provide methods for the desired interaction with a NVMe controller / adapter. Best regards, Robert. From: Robles, Raymond C [mailto:raymond.c.robles at intel.com] Sent: Thursday, November 07, 2013 3:29 PM To: Robert Randall (rrandall); nvmewin at lists.openfabrics.org Cc: Mayes, Barrett N Subject: RE: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges Hi Robert, Microsoft has been very adamant about not supporting pass through IOCTLs (via IOCTL_SCSI_MINIPORT) and the potential security threat this mechanism poses. There is a potential for them to listen to suggestions for workarounds, or alternatives, to pass through IOCTLs. However, they have been very clear about not supporting pass through IOCTLs. Thanks, Ray From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Robert Randall (rrandall) Sent: Thursday, November 07, 2013 2:18 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges Hello all, We have been testing the NVMe in-box driver included in Windows 8.1 and Server 2012 R2. It does not appear to have any IOCTL support for ADMIN commands. This is very sad. This renders the in-box driver useless for uses cases where VU ADMIN commands are used to, for example, manage NVM Express Namespaces. Have others had a similar experience with the in-box driver? Microsoft appears willing to listen to requests to add capabilities to the in-box driver. This represents an opportunity for all of us participating in the NVM Express ecosystem to express our requirements to Microsoft with one voice. If you believe that the in-box driver is lacking critical features (as I do) I would be happy to coordinate a unified response to the Microsoft program manager responsible for the in-box device driver. Perhaps this would be a useful topic for the next technical conference call? Best regards, Robert Robert Randall mobile: 612.770.9612 rrandall at micron.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From akyros000 at gmail.com Fri Nov 8 07:03:41 2013 From: akyros000 at gmail.com (Jeff Glass) Date: Fri, 08 Nov 2013 07:03:41 -0800 Subject: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - some challenges In-Reply-To: <70C73440F9F7C24F81A11355C292A9B5C10FFCBF@NTXBOIMBX04.micron.com> References: <70C73440F9F7C24F81A11355C292A9B5C10FF73C@NTXBOIMBX04.micron.com> <49158E750348AA499168FD41D8898360625F3BE9@FMSMSX105.amr.corp.intel.com> <70C73440F9F7C24F81A11355C292A9B5C10FFCBF@NTXBOIMBX04.micron.com> Message-ID: <527CFD4D.7010905@gmail.com> Microsoft's position on this seems odd to me. I don't think it's good for NVMe or Microsoft. If they're not providing a pass through interface, then people are either going to 1. Ship their own drivers so they can do things like create/delete/modify namespaces 2. Need to ship configuration utility on a separate linux live CD to do those types of things. I too understand their position, but the solution they've delivered is going to result in a bad user experience. On 11/8/2013 6:20 AM, Robert Randall (rrandall) wrote: > > Hi Ray, > > > > Well stated. However, I confess that I find it odd that Microsoft > would not consider providing a WMI interface. I believe that a WMI > interface would satisfy their security concerns. Has this been > discussed with Microsoft? I would be happy to provide a simple > definition of a WMI class which would provide methods for the desired > interaction with a NVMe controller / adapter. > > > > Best regards, > > Robert. > > > > *From:*Robles, Raymond C [mailto:raymond.c.robles at intel.com] > *Sent:* Thursday, November 07, 2013 3:29 PM > *To:* Robert Randall (rrandall); nvmewin at lists.openfabrics.org > > *Cc:* Mayes, Barrett N > *Subject:* RE: [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - > some challenges > > > > Hi Robert, > > > > Microsoft has been very adamant about not supporting pass through > IOCTLs (via IOCTL_SCSI_MINIPORT) and the potential security threat > this mechanism poses. There is a potential for them to listen to > suggestions for workarounds, or alternatives, to pass through IOCTLs. > However, they have been very clear about not supporting pass through > IOCTLs. > > > > Thanks, > > Ray > > > > *From:* nvmewin-bounces at lists.openfabrics.org > > [mailto:nvmewin-bounces at lists.openfabrics.org] *On Behalf Of *Robert > Randall (rrandall) > *Sent:* Thursday, November 07, 2013 2:18 PM > *To:* nvmewin at lists.openfabrics.org > *Subject:* [nvmewin] NVMe in-box driver in Win8.1/Server 2012 R2 - > some challenges > > > > Hello all, > > > > We have been testing the NVMe in-box driver included in Windows 8.1 > and Server 2012 R2. It does not appear to have any IOCTL support for > ADMIN commands. This is very sad. This renders the in-box driver > useless for uses cases where VU ADMIN commands are used to, for > example, manage NVM Express Namespaces. > > > > Have others had a similar experience with the in-box driver? > > > > Microsoft appears willing to listen to requests to add capabilities to > the in-box driver. This represents an opportunity for all of us > participating in the NVM Express ecosystem to express our requirements > to Microsoft with one voice. > > > > If you believe that the in-box driver is lacking critical features (as > I do) I would be happy to coordinate a unified response to the > Microsoft program manager responsible for the in-box device driver. > > > > Perhaps this would be a useful topic for the next technical conference > call? > > > > Best regards, > > Robert > > > > Robert Randall > > mobile: 612.770.9612 > > rrandall at micron.com > > > > > > _______________________________________________ > nvmewin mailing list > nvmewin at lists.openfabrics.org > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Fri Nov 8 11:11:01 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 8 Nov 2013 19:11:01 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: <1fcd76855c0847dcb32cbdf7a4ac82cc@DM2PR07MB285.namprd07.prod.outlook.com> References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> <1fcd76855c0847dcb32cbdf7a4ac82cc@DM2PR07MB285.namprd07.prod.outlook.com> Message-ID: Hi Carolyn, Have you had a chance to review/test the patch? It'd be fine if you need more time. Please advise when we will receive approval from INTEL. Thanks a lot, Alex From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Wednesday, November 06, 2013 1:20 PM To: Alex Chang; Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, We are good with this change. Thanks, -Rick From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 12:01 PM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Great. Thanks. I will do more testing while waiting the approvals from LSI and Intel. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, November 06, 2013 11:57 AM To: Alex Chang; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From carolyn.d.foster at intel.com Fri Nov 8 11:52:09 2013 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Fri, 8 Nov 2013 19:52:09 +0000 Subject: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 In-Reply-To: References: <31536_1378920019_5230A653_31536_519_1_23EC73C80FB59046A6B7B8EB7B382659320938BB@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA62A@SACMBXIP01.sdcorp.global.sandisk.com> <55ecd7eb9b044e309b4d8dbd4c55c5c2@DM2PR07MB285.namprd07.prod.outlook.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAA66C@SACMBXIP01.sdcorp.global.sandisk.com> <23EC73C80FB59046A6B7B8EB7B3826593BDAC19E@SACMBXIP01.sdcorp.global.sandisk.com> <1fcd76855c0847dcb32cbdf7a4ac82cc@DM2PR07MB285.namprd07.prod.outlook.com> Message-ID: Hi Alex, I apologize for the delay, I will provide feedback by the end of next week, November 15th. Will that work? Thank you! Carolyn From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Friday, November 08, 2013 12:11 PM To: Knoblaugh, Rick; Dharani Kotte; nvmewin at lists.openfabrics.org; Foster, Carolyn D Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Carolyn, Have you had a chance to review/test the patch? It'd be fine if you need more time. Please advise when we will receive approval from INTEL. Thanks a lot, Alex From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Wednesday, November 06, 2013 1:20 PM To: Alex Chang; Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, We are good with this change. Thanks, -Rick From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 12:01 PM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Great. Thanks. I will do more testing while waiting the approvals from LSI and Intel. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, November 06, 2013 11:57 AM To: Alex Chang; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, Yes, I did. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, November 06, 2013 11:52 AM To: Dharani Kotte; Knoblaugh, Rick; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I had done the testing on both Windows 7 and 8, 32-bit version via drive formatting, IOMeters, SCSI Compliance and SDStress. Since your change involves queue sharing among multiple cores, did you verify it in the sharing situation? Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Tuesday, October 29, 2013 1:21 PM To: Knoblaugh, Rick; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Nothing has changed from that code but just added the MultipleCoresToSingleQueueFlag in one more place that is suggested by Alex. That is the only change after the review. Thanks, Dharani. From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Tuesday, October 29, 2013 12:27 PM To: Dharani Kotte; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, I recall reviewing those changes involving MultipleCoresToSingleQueueFlag back at end of July. We were fine with that code. It's been a while. If nothing has changed since then, looks good. (If anything tweaked since July, can you please resend?) Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Tuesday, October 29, 2013 11:28 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The changes are very limited, files nvmeInit.c, nvmeStd.c, nvmeStd.h and nvme.inf has the changes. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 29, 2013 11:01 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi all, I'd like to remind you all to review/test the patch submitted by Dharani for 32-bit support. Hopefully, we can complete the review in two weeks and, after receiving approvals from Intel and LSI, I can push it then. Please provide feedbacks and verify it works fine while testing it. Hi Dharani, Thanks a lot for the effort. I have a quick question on your changes: is there any specific modifications you added that's related to 32-bit support? Regards, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, September 11, 2013 10:20 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin I have added the flag in the necessary place. Run the SDStress/SCSI compliance on both Win7/Win8 32-bit systems. The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Thursday, August 08, 2013 9:37 AM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please find the attached SDStress.zip, which includes the required binaries and a batch file for your reference. Before running it, you need to create a volume and its drive letter is passed as first parameter to the batch file, for example: sdstress.bat E Please let me know if you have any questions. Thanks for your help. Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Thursday, August 08, 2013 9:27 AM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, I will add the flag where ever necessary. Can you please send me the SDStress application that you are using so that I can perform this test as well. Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:26 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, When you said "it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement.", which is not true. And you're right that there will be one IO queue allocation attempt at least. Looks like whenever any given queue is shared by multiple cores, you need to set the flag to be ture. Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 5:16 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex. Comments in Green ... Thanks, Dharani. ________________________________ From: Alex Chang [Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 5:06 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Please see my comments in red... Thanks, Alex From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, August 07, 2013 4:00 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Alex, The situation was that when "pQI->NumSubIoQAllocFromAdapter" variable is set to 1 which means that the controller want only 1 IO queue. In this case it doesn't even enter into "if (pQI->NumSubIoQAllocated < QueueID){}" statement. If that's true, then there is no IO queue allocated at all, isn't it? 1 Queue is created but after that the Queue number will be forced to 1 and hence all the cores available are getting mapped to Queue 1. But we need to use StartLocks when ever 2 or more cores shares the same queue. The "fall back to use only one queue" case is covered only in the situation where the NVMeAllocQueues() fails for the Queue, This condition I didn't hit but you are right we may have to add this flag in that condition as well. drive formatting - If it is windows NTFS format I tested for Win8/Win7 32-bit. IOMeter - I tested for Win8/Win7 32-bit SDStress - I have not tested is it part of WHQL. Please test it if you have chance. I don't have a WHQL setup right now, Can you pass the SDStress exe if anything available with you. SCSI compliance - I have not tested since I didn't touch any part of the code that is related to SCSI Compliance but I can test it Thanks, Dharani. From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, August 07, 2013 3:46 PM To: Dharani Kotte; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Hi Dharani, Sorry to take such long time to get back to you and thank you for the effort. I just finished reviewing your changes and noticed you added a flag called "MultiplecoresToSingleQueueFlag". Does it mean there is only one IO queue being allocated and shared by all the cores? If yes, then the line you added (line#2828) in nvmeinit.c should be moved up to "fall back to use only one queue" case. Let me know what you think. By the way, have you tests it with drive formatting, IOMeter, SDStress and SCSI compliance yet? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Dharani Kotte Sent: Wednesday, July 31, 2013 9:53 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] ***UNCHECKED*** [WARNING - ENCRYPTED ATTACHMENT NOT VIRUSSCANNED] OFA NVMe Windows driver contribution - 32-bitsupport Win7/Win8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Please review it and let me know the comments. Password: sndk1234 Thanks, Dharani. From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Wednesday, July 31, 2013 9:29 AM To: Dharani Kotte Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: RE: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Dharani, This is great. Would you please email it out to the following mailing list asking for review and approval ? nvmewin at lists.openfabrics.org thanks -Kwok From: Dharani Kotte [mailto:Dharani.Kotte at sandisk.com] Sent: Wednesday, July 31, 2013 9:08 AM To: Kong, Kwok (Kwok.Kong at idt.com); Kwok Kong Cc: Dave Landsman; Gurpreet Anand; Sumant Patro Subject: [WARNING - ENCRYPTED ATTACHMENT NOT VIRUS SCANNED] OFA NVMe Windows driver contribution Hi Kwok, The attached is the code with 32-bit support tested on the Win7/Win8 32-bit systems. Can you please send it for review and let me know the comments. Password: sndk1234 Thanks, Dharani. -----Original Message----- From: Kong, Kwok [mailto:Kwok.Kong at idt.com] Sent: Wednesday, June 26, 2013 12:58 To: Dave Landsman Subject: OFA NVMe Windows driver contribution Dave, We are working on a NVMe Windows driver feature planning for the Dec 2013 release. Samsung cannot take on the task to get the driver to support windows 32-bit systems. I wonder if Sandisk can take on the task to get the driver to work in Windows 32-bit system. It is much appreciated if Sandisk can take this task on. Please let me know what you think. Thanks -Kwok ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Fri Nov 15 09:11:08 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 15 Nov 2013 17:11:08 +0000 Subject: [nvmewin] NVMe Windows DB Is LOCKED - Pushing Patch From SanDisk For 32-bit Windows Support Message-ID: Locking NVMe Windows DB. Thanks, Alex nvmewin mailing list nvmewin at lists.openfabrics.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Fri Nov 15 09:57:13 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 15 Nov 2013 17:57:13 +0000 Subject: [nvmewin] NVMe Windows DB Is UNLOCKED - Pushing Patch From SanDisk For 32-bit Windows Support Finished Message-ID: Hi all, Thank you, Dharani, for the effort. The patch had been updated to the source base and a new tag called "32-bit_Windows_support" had been created under tags directory. Should you have any questions, please reply to the email listed below. Thanks, Alex nvmewin mailing list nvmewin at lists.openfabrics.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From Dharani.Kotte at sandisk.com Fri Nov 15 10:49:28 2013 From: Dharani.Kotte at sandisk.com (Dharani Kotte) Date: Fri, 15 Nov 2013 18:49:28 +0000 Subject: [nvmewin] NVMe Windows DB Is UNLOCKED - Pushing Patch From SanDisk For 32-bit Windows Support Finished In-Reply-To: References: Message-ID: <23EC73C80FB59046A6B7B8EB7B3826593BDB0BB1@SACMBXIP01.sdcorp.global.sandisk.com> My pleasure, Thank you for your time in review and accepting the Patch. Thank, Dharani. From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Friday, November 15, 2013 9:57 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows DB Is UNLOCKED - Pushing Patch From SanDisk For 32-bit Windows Support Finished Hi all, Thank you, Dharani, for the effort. The patch had been updated to the source base and a new tag called "32-bit_Windows_support" had been created under tags directory. Should you have any questions, please reply to the email listed below. Thanks, Alex nvmewin mailing list nvmewin at lists.openfabrics.org ________________________________ PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -------------- next part -------------- An HTML attachment was scrubbed... URL: From Kwok.Kong at pmcs.com Fri Nov 15 16:03:28 2013 From: Kwok.Kong at pmcs.com (Kwok Kong) Date: Sat, 16 Nov 2013 00:03:28 +0000 Subject: [nvmewin] Quarterly Status meeting Message-ID: <03D88B383FA04244AA514AA931F7B1290CB9CE4B@BBYEXM01.pmc-sierra.internal> When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: - Review 1.3 release plan o Windows 32-bit support o Windows 8 extended SRB support o NVM format state machine enhancement o End to end protection support o Hibernation - Known issues o - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) o NUMA group support in core enumeration o Controller reset does not handle all cases o Learning of CPU core to Vector failure handling o Core-MSI vector queue mapping issues - Discuss plan for 2014 - Features that will not be supported in 2013 - =========================================== - - NVMe 1.1 support: - - multi-path - - SGL - - Get/Set feature update - - Autonomous power state transition - - Host Identifier - - Reservation Notification Mask - - Reservation Persistence - - identify structure update - - write zeros command - Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft® Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/calendar Size: 5072 bytes Desc: not available URL: From Yong.sc.Chen at huawei.com Mon Nov 18 16:13:17 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Tue, 19 Nov 2013 00:13:17 +0000 Subject: [nvmewin] code review: crash dump & hibernation support Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE51482@sjceml501-mbs.china.huawei.com> Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo] Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: testlog.zip Type: application/x-zip-compressed Size: 120219 bytes Desc: testlog.zip URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: source-codereview-1118.zip Type: application/x-zip-compressed Size: 176885 bytes Desc: source-codereview-1118.zip URL: From Yong.sc.Chen at huawei.com Tue Nov 19 15:59:16 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Tue, 19 Nov 2013 23:59:16 +0000 Subject: [nvmewin] code review: crash dump & hibernation support Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 6737 bytes Desc: image002.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: testlog.zip Type: application/x-zip-compressed Size: 120219 bytes Desc: testlog.zip URL: -------------- next part -------------- An embedded message was scrubbed... From: etranssupport Subject: 【来自于华为公司】请您下载文件，No：800404158 . [From Huawei] Please download the file from the eTrans application No: 800404158 Date: Tue, 19 Nov 2013 02:38:39 +0000 Size: 9009 URL: From Alex.Chang at pmcs.com Tue Nov 19 16:05:13 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 20 Nov 2013 00:05:13 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> Message-ID: Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: source-hiber-1118.zip Type: application/x-zip-compressed Size: 176885 bytes Desc: source-hiber-1118.zip URL: From Yong.sc.Chen at huawei.com Wed Nov 20 00:51:37 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Wed, 20 Nov 2013 08:51:37 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE529A8@sjceml501-mbs.china.huawei.com> Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: ・ nvmeInit.c changes to manage buffers and IO queues in dump modes. ・ nvmeIo.c minor tweak in dump mode where only boot-CPU is available ・ nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). ・ nvmeStat.c helper function change due to some timer related API not available in dump mode. ・ nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 6737 bytes Desc: image002.jpg URL: From santosh.s2 at samsung.com Thu Nov 21 23:08:42 2013 From: santosh.s2 at samsung.com (Santosh Singh) Date: Fri, 22 Nov 2013 12:38:42 +0530 Subject: [nvmewin] Performance issue Message-ID: <01d601cee751$addde210$0999a630$@samsung.com> Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: . Review 1.3 release plan . Windows 32-bit support . Windows 8 extended SRB support . NVM format state machine enhancement . End to end protection support . Hibernation . Known issues . - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) . NUMA group support in core enumeration . Controller reset does not handle all cases . Learning of CPU core to Vector failure handling . Core-MSI vector queue mapping issues . Discuss plan for 2014 . Features that will not be supported in 2013 . =========================================== . . NVMe 1.1 support: . - multi-path . - SGL . - Get/Set feature update . - Autonomous power state transition . - Host Identifier . - Reservation Notification Mask . - Reservation Persistence . - identify structure update . - write zeros command . Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using MicrosoftR Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: . I am connecting from inside the PMC-Sierra network . I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492 ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: . Inside the PMC-Sierra network . Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From judy.brock at ssi.samsung.com Fri Nov 22 00:52:51 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Fri, 22 Nov 2013 08:52:51 +0000 Subject: [nvmewin] Performance issue In-Reply-To: <01d601cee751$addde210$0999a630$@samsung.com> References: <01d601cee751$addde210$0999a630$@samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> Per the attached excerpts of archived nvmewin reflector exchanges, apparently this line of code was discovered to have been causing significant performance problems a year ago and was to have been removed until the issue was understood/resolved. Somehow it is still in the current code base and causing (us at least) the same drastic performance problems that were reported by others as far back as last November. I'm confused as to how this was resolved for others since the code is still there.... Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Santosh Singh Sent: Thursday, November 21, 2013 11:09 PM To: nvmewin at lists.openfabrics.org Cc: 'Kwok Kong' Subject: [nvmewin] Performance issue Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded message was scrubbed... From: "Luse, Paul E" Subject: Re: [nvmewin] Bug Fix Patch - Review Request Date: Wed, 7 Nov 2012 23:21:13 +0000 Size: 69198 URL: From Kwok.Kong at pmcs.com Fri Nov 22 08:16:49 2013 From: Kwok.Kong at pmcs.com (Kwok Kong) Date: Fri, 22 Nov 2013 16:16:49 +0000 Subject: [nvmewin] Performance issue In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> References: <01d601cee751$addde210$0999a630$@samsung.com> <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> Message-ID: <03D88B383FA04244AA514AA931F7B1290CBA3CCA@BBYEXM01.pmc-sierra.internal> Let me add this to the agenda for next week's call. -Kwok From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Friday, November 22, 2013 12:53 AM To: Santosh Singh; nvmewin at lists.openfabrics.org Cc: Kwok Kong Subject: RE: [nvmewin] Performance issue Per the attached excerpts of archived nvmewin reflector exchanges, apparently this line of code was discovered to have been causing significant performance problems a year ago and was to have been removed until the issue was understood/resolved. Somehow it is still in the current code base and causing (us at least) the same drastic performance problems that were reported by others as far back as last November. I'm confused as to how this was resolved for others since the code is still there.... Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Santosh Singh Sent: Thursday, November 21, 2013 11:09 PM To: nvmewin at lists.openfabrics.org Cc: 'Kwok Kong' Subject: [nvmewin] Performance issue Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Fri Nov 22 09:33:02 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 22 Nov 2013 17:33:02 +0000 Subject: [nvmewin] Performance issue In-Reply-To: <03D88B383FA04244AA514AA931F7B1290CBA3CCA@BBYEXM01.pmc-sierra.internal> References: <01d601cee751$addde210$0999a630$@samsung.com> <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> <03D88B383FA04244AA514AA931F7B1290CBA3CCA@BBYEXM01.pmc-sierra.internal> Message-ID: Hi Rick, The line was included in your TRIM support patch last June. Could you please elaborate why you added it? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Friday, November 22, 2013 8:17 AM To: Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Let me add this to the agenda for next week's call. -Kwok From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Friday, November 22, 2013 12:53 AM To: Santosh Singh; nvmewin at lists.openfabrics.org Cc: Kwok Kong Subject: RE: [nvmewin] Performance issue Per the attached excerpts of archived nvmewin reflector exchanges, apparently this line of code was discovered to have been causing significant performance problems a year ago and was to have been removed until the issue was understood/resolved. Somehow it is still in the current code base and causing (us at least) the same drastic performance problems that were reported by others as far back as last November. I'm confused as to how this was resolved for others since the code is still there.... Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Santosh Singh Sent: Thursday, November 21, 2013 11:09 PM To: nvmewin at lists.openfabrics.org Cc: 'Kwok Kong' Subject: [nvmewin] Performance issue Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Rick.Knoblaugh at lsi.com Fri Nov 22 11:33:19 2013 From: Rick.Knoblaugh at lsi.com (Knoblaugh, Rick) Date: Fri, 22 Nov 2013 19:33:19 +0000 Subject: [nvmewin] Performance issue In-Reply-To: References: <01d601cee751$addde210$0999a630$@samsung.com> <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> <03D88B383FA04244AA514AA931F7B1290CBA3CCA@BBYEXM01.pmc-sierra.internal> Message-ID: Hi Alex, Yes, the call to StorPortGetUncachedExtension was included with the initial TRIM code, so that folks evaluating the TRIM patch would not immediately hit the known assert issue. I went back and took a look at the release notes I had sent out with the patch: In October of 2012, a patch had been put in NVMeFindAdapter to get around an assert that was occurring on checked builds of Windows 8. It was subsequently removed as it was determined there may be some performance hit. Now that we are specifically testing Windows 8 features, it has been put back here as a convenience to avoid getting the assert right out the chute. This can be taken out again if so desired. Given the performance issue, this should probably be again removed from the code base. Greatly appreciate you guys revisiting this topic. Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Friday, November 22, 2013 9:33 AM To: Kwok Kong; Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Hi Rick, The line was included in your TRIM support patch last June. Could you please elaborate why you added it? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Friday, November 22, 2013 8:17 AM To: Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Let me add this to the agenda for next week's call. -Kwok From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Friday, November 22, 2013 12:53 AM To: Santosh Singh; nvmewin at lists.openfabrics.org Cc: Kwok Kong Subject: RE: [nvmewin] Performance issue Per the attached excerpts of archived nvmewin reflector exchanges, apparently this line of code was discovered to have been causing significant performance problems a year ago and was to have been removed until the issue was understood/resolved. Somehow it is still in the current code base and causing (us at least) the same drastic performance problems that were reported by others as far back as last November. I'm confused as to how this was resolved for others since the code is still there.... Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Santosh Singh Sent: Thursday, November 21, 2013 11:09 PM To: nvmewin at lists.openfabrics.org Cc: 'Kwok Kong' Subject: [nvmewin] Performance issue Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Fri Nov 22 11:37:45 2013 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 22 Nov 2013 19:37:45 +0000 Subject: [nvmewin] Performance issue In-Reply-To: References: <01d601cee751$addde210$0999a630$@samsung.com> <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> <03D88B383FA04244AA514AA931F7B1290CBA3CCA@BBYEXM01.pmc-sierra.internal> Message-ID: Thank you very much, Rick. With the details, we can discuss more in the meeting on Monday to determine what to do with it. Alex From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Friday, November 22, 2013 11:33 AM To: Alex Chang; Kwok Kong; Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] Performance issue Hi Alex, Yes, the call to StorPortGetUncachedExtension was included with the initial TRIM code, so that folks evaluating the TRIM patch would not immediately hit the known assert issue. I went back and took a look at the release notes I had sent out with the patch: In October of 2012, a patch had been put in NVMeFindAdapter to get around an assert that was occurring on checked builds of Windows 8. It was subsequently removed as it was determined there may be some performance hit. Now that we are specifically testing Windows 8 features, it has been put back here as a convenience to avoid getting the assert right out the chute. This can be taken out again if so desired. Given the performance issue, this should probably be again removed from the code base. Greatly appreciate you guys revisiting this topic. Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Friday, November 22, 2013 9:33 AM To: Kwok Kong; Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Hi Rick, The line was included in your TRIM support patch last June. Could you please elaborate why you added it? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Friday, November 22, 2013 8:17 AM To: Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Let me add this to the agenda for next week's call. -Kwok From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Friday, November 22, 2013 12:53 AM To: Santosh Singh; nvmewin at lists.openfabrics.org Cc: Kwok Kong Subject: RE: [nvmewin] Performance issue Per the attached excerpts of archived nvmewin reflector exchanges, apparently this line of code was discovered to have been causing significant performance problems a year ago and was to have been removed until the issue was understood/resolved. Somehow it is still in the current code base and causing (us at least) the same drastic performance problems that were reported by others as far back as last November. I'm confused as to how this was resolved for others since the code is still there.... Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Santosh Singh Sent: Thursday, November 21, 2013 11:09 PM To: nvmewin at lists.openfabrics.org Cc: 'Kwok Kong' Subject: [nvmewin] Performance issue Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Yong.sc.Chen at huawei.com Fri Nov 22 13:47:37 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Fri, 22 Nov 2013 21:47:37 +0000 Subject: [nvmewin] code review: crash dump & hibernation support References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I’d very appreciate it. Holiday is upon us and we’d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: ・ nvmeInit.c changes to manage buffers and IO queues in dump modes. ・ nvmeIo.c minor tweak in dump mode where only boot-CPU is available ・ nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). ・ nvmeStat.c helper function change due to some timer related API not available in dump mode. ・ nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From judy.brock at ssi.samsung.com Fri Nov 22 16:02:00 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Sat, 23 Nov 2013 00:02:00 +0000 Subject: [nvmewin] Performance issue In-Reply-To: References: <01d601cee751$addde210$0999a630$@samsung.com> <36E8D38D6B771A4BBDB1C0D800158A514E46BD7C@SSIEXCH-MB3.ssi.samsung.com> <03D88B383FA04244AA514AA931F7B1290CBA3CCA@BBYEXM01.pmc-sierra.internal> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46C098@SSIEXCH-MB3.ssi.samsung.com> >> "It was subsequently removed as it was determined there may be some performance hit." For future reference, I would ask that we add a much more explicit detailed warnings if bugs of a severe nature such as a 50% reduction in performance are re-introduced deliberately. Imo the sentence above does not adequately describe what we ran into and what took a lot of effort to isolate - unless one had been around as an active participant on the in October of 2012, one would not have known at all what to look for. Thanks, Judy From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Friday, November 22, 2013 11:38 AM To: Knoblaugh, Rick; Kwok Kong; Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] Performance issue Thank you very much, Rick. With the details, we can discuss more in the meeting on Monday to determine what to do with it. Alex From: Knoblaugh, Rick [mailto:Rick.Knoblaugh at lsi.com] Sent: Friday, November 22, 2013 11:33 AM To: Alex Chang; Kwok Kong; Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] Performance issue Hi Alex, Yes, the call to StorPortGetUncachedExtension was included with the initial TRIM code, so that folks evaluating the TRIM patch would not immediately hit the known assert issue. I went back and took a look at the release notes I had sent out with the patch: In October of 2012, a patch had been put in NVMeFindAdapter to get around an assert that was occurring on checked builds of Windows 8. It was subsequently removed as it was determined there may be some performance hit. Now that we are specifically testing Windows 8 features, it has been put back here as a convenience to avoid getting the assert right out the chute. This can be taken out again if so desired. Given the performance issue, this should probably be again removed from the code base. Greatly appreciate you guys revisiting this topic. Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Friday, November 22, 2013 9:33 AM To: Kwok Kong; Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Hi Rick, The line was included in your TRIM support patch last June. Could you please elaborate why you added it? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Friday, November 22, 2013 8:17 AM To: Judy Brock-SSI; Santosh Singh; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] Performance issue Let me add this to the agenda for next week's call. -Kwok From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Friday, November 22, 2013 12:53 AM To: Santosh Singh; nvmewin at lists.openfabrics.org Cc: Kwok Kong Subject: RE: [nvmewin] Performance issue Per the attached excerpts of archived nvmewin reflector exchanges, apparently this line of code was discovered to have been causing significant performance problems a year ago and was to have been removed until the issue was understood/resolved. Somehow it is still in the current code base and causing (us at least) the same drastic performance problems that were reported by others as far back as last November. I'm confused as to how this was resolved for others since the code is still there.... Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Santosh Singh Sent: Thursday, November 21, 2013 11:09 PM To: nvmewin at lists.openfabrics.org Cc: 'Kwok Kong' Subject: [nvmewin] Performance issue Hi Kwok/Alex, There is a performance issue in our OFA driver on Windows 2012 and same should be applicable to Windows 8. In the NVMeFindAdapter() function , code specifically written for the Windows 8 under the NTDDI_VERSION > NTDDI_WIN7 . #if (NTDDI_VERSION > NTDDI_WIN7) /* Ensure Storport allocates the DMA adapter object */ StorPortGetUncachedExtension(pAE, pPCI, 1); #endif After removing this code the performance on the multi core system increases 2 times on window 2k12 , and removing the 1 Byte allocation does not have any side effect on the driver functionality. As per the Storport documentation the StorPortGetUncachedExtension routine allocates an uncached common buffer to be shared by the CPU and the device. Not sure but my best guess is that buffer sharing among CPUs could be a reason of low performance. Can we add this in the known issue. Regards Santosh -----Original Appointment----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Kwok Kong Sent: Saturday, November 16, 2013 5:33 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Note: The GMT offset above does not reflect daylight saving time adjustments. *~*~*~*~*~*~*~*~*~* Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: Untitled attachment 00195.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From judy.brock at ssi.samsung.com Sun Nov 24 06:54:03 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Sun, 24 Nov 2013 14:54:03 +0000 Subject: [nvmewin] Quarterly Status meeting In-Reply-To: <03D88B383FA04244AA514AA931F7B1290CB9CE4B@BBYEXM01.pmc-sierra.internal> References: <03D88B383FA04244AA514AA931F7B1290CB9CE4B@BBYEXM01.pmc-sierra.internal> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46C2DC@SSIEXCH-MB3.ssi.samsung.com> Kwok, Here are some additional items to add to the agenda: 1. BUILDIO - a. there are some IOCTLs which wind up launching actual IO from the BuildIo context. This should never happen - they need to stage for the IO op and then launch their state machines in StartIo. 2. MIGRATE to VS2013/WDK 8.1 a. Microsoft came out with VS2013/WDK 8.1 many months ago at this point. b. Migrating to the VS2013/WDK 8.1 platform will allow us to take advantage of StorPortGetProcessorIndexFromNumber(). This API makes fixing enumeration/organization of core table array (and subsequent runtime lookups) much easier. 3. NVM FMT CMD - a. In addition to needing to enhance the NVM Format state machine -already an agenda item - (to quiesce all outstanding IO before initiating the format/reject new reqs while cmd is in progress, etc) there are also bugs in the logic when the Global Identifier is used (I don't remember what they are, I just remember running into them around the time of 1st plugfest) 4. PRP LIST BUILDING - a. Function NVMePreparePRPs() never fleshes out the PRP List for NVMe commands that need more than 2 PRP entries. So for ex, if a Vendor Specific command is sent via private IOCTL and the 2nd PRP entry is a pointer to a PRP list, the list is never built at all. The routine needs to be expanded to handle more than 2 PRP entries. 5. PARAMLISTLENGTH - a. In SntiTranslateWriteBuffer (and maybe other routines, need to look again), "dword10 |= paramListLength;" should be changed to "dword10 |= paramListLength / NUM_BYTES_IN_DWORDS;" where NUM_BYTES_IN_DWORDS == 4 6. ORPHANED REQUESTS - a. FormatNVMFailure() still leaving orphaned requests outstanding if it encounters an error. This will cause an application that sent the request to hang waiting for a completion that will never come. I sent a proposed patch to the reflector many months ago on the subject. 7. FreeQList ACCESSES - a. In the StartIo path, protect manipulation of SQI->FreeQList in ProcessIo from manipulation of the same list in IoCompletion DPC. Recommend taking DPC lock to do so. (It's possible that adding group-awareness to core table array will eliminate observation of simultaneous threads of execution in both code paths - needs more investigation). Thanks, Judy -----Original Appointment----- From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Friday, November 15, 2013 4:03 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: ATT00001.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From judy.brock at ssi.samsung.com Sun Nov 24 08:03:59 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Sun, 24 Nov 2013 16:03:59 +0000 Subject: [nvmewin] Quarterly Status meeting References: <03D88B383FA04244AA514AA931F7B1290CB9CE4B@BBYEXM01.pmc-sierra.internal> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46C329@SSIEXCH-MB3.ssi.samsung.com> Forgot to mention the following bug: NVMeInitAdminQueues(pAE)returns TRUE/FALSE. However, callers of that routine are checking for return value of STOR_STATUS_SUCCESS. The function should be changed to "return (Status)" in several places. Thanks, Judy _____________________________________________ From: Judy Brock-SSI Sent: Sunday, November 24, 2013 6:54 AM To: 'Kwok Kong'; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] Quarterly Status meeting Kwok, Here are some additional items to add to the agenda: 1. BUILDIO - a. there are some IOCTLs which wind up launching actual IO from the BuildIo context. This should never happen - they need to stage for the IO op and then launch their state machines in StartIo. 2. MIGRATE to VS2013/WDK 8.1 a. Microsoft came out with VS2013/WDK 8.1 many months ago at this point. b. Migrating to the VS2013/WDK 8.1 platform will allow us to take advantage of StorPortGetProcessorIndexFromNumber(). This API makes fixing enumeration/organization of core table array (and subsequent runtime lookups) much easier. 3. NVM FMT CMD - a. In addition to needing to enhance the NVM Format state machine -already an agenda item - (to quiesce all outstanding IO before initiating the format/reject new reqs while cmd is in progress, etc) there are also bugs in the logic when the Global Identifier is used (I don't remember what they are, I just remember running into them around the time of 1st plugfest) 4. PRP LIST BUILDING - a. Function NVMePreparePRPs() never fleshes out the PRP List for NVMe commands that need more than 2 PRP entries. So for ex, if a Vendor Specific command is sent via private IOCTL and the 2nd PRP entry is a pointer to a PRP list, the list is never built at all. The routine needs to be expanded to handle more than 2 PRP entries. 5. PARAMLISTLENGTH - a. In SntiTranslateWriteBuffer (and maybe other routines, need to look again), "dword10 |= paramListLength;" should be changed to "dword10 |= paramListLength / NUM_BYTES_IN_DWORDS;" where NUM_BYTES_IN_DWORDS == 4 6. ORPHANED REQUESTS - a. FormatNVMFailure() still leaving orphaned requests outstanding if it encounters an error. This will cause an application that sent the request to hang waiting for a completion that will never come. I sent a proposed patch to the reflector many months ago on the subject. 7. FreeQList ACCESSES - a. In the StartIo path, protect manipulation of SQI->FreeQList in ProcessIo from manipulation of the same list in IoCompletion DPC. Recommend taking DPC lock to do so. (It's possible that adding group-awareness to core table array will eliminate observation of simultaneous threads of execution in both code paths - needs more investigation). Thanks, Judy -----Original Appointment----- From: Kwok Kong [mailto:Kwok.Kong at pmcs.com] Sent: Friday, November 15, 2013 4:03 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] Quarterly Status meeting When: Monday, November 25, 2013 1:00 PM-2:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: Live Meeting Agenda: * Review 1.3 release plan * Windows 32-bit support * Windows 8 extended SRB support * NVM format state machine enhancement * End to end protection support * Hibernation * Known issues * - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset. (Judy - Samsung) * NUMA group support in core enumeration * Controller reset does not handle all cases * Learning of CPU core to Vector failure handling * Core-MSI vector queue mapping issues * Discuss plan for 2014 * Features that will not be supported in 2013 * =========================================== * * NVMe 1.1 support: * - multi-path * - SGL * - Get/Set feature update * - Autonomous power state transition * - Host Identifier * - Reservation Notification Mask * - Reservation Persistence * - identify structure update * - write zeros command * Next Meeting Please let me know if you would like to add any items to the agenda. -Kwok -+-----+-----+-----+-----+-----+-----+-----+-----+- Kwok Kong has invited you to attend an online meeting using Microsoft(r) Office Communications Server. Join the meeting Make sure the Office Live Meeting client is installed before the meeting: * I am connecting from inside the PMC-Sierra network * I am connecting from outside the PMC-Sierra network AUDIO INFORMATION To join a meeting from your phone, dial in using the following information: Phone: Burnaby Ext 6026 [English, French] Phone: +1 (888) 828-7722 [English, Spanish, French] Phone: +1 (604) 415-6026 [English, French] Find a local phone number for your region Conference ID: 8339027 Passcode: Passcode is not required. Note: If you have an account on this corporate network, use your PIN to join. Have you set your PIN? TROUBLESHOOTING Unable to join the meeting? Start Office Live Meeting and join the meeting with the following information: Meeting ID: bd6164cdeb094e6492ca14408eca1dd3 Entry Code: 8246 Location: meet:sip:kwok.kong at pmcs.com;gruu;opaque=app:conf:focus:id:bd6164cdeb094e6492ca14408eca1dd3%3Fconf-key=8246 If you still cannot enter the meeting, contact support: * Inside the PMC-Sierra network * Outside the PMC-Sierra network NOTICE Office Live Meeting can be used to record meetings. By participating in this meeting, you agree that your communications may be monitored or recorded at any time during the meeting. << File: ATT00001.txt >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From judy.brock at ssi.samsung.com Sun Nov 24 08:19:45 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Sun, 24 Nov 2013 16:19:45 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ‘1’. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn’t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I’d very appreciate it. Holiday is upon us and we’d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: ・ nvmeInit.c changes to manage buffers and IO queues in dump modes. ・ nvmeIo.c minor tweak in dump mode where only boot-CPU is available ・ nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). ・ nvmeStat.c helper function change due to some timer related API not available in dump mode. ・ nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 6737 bytes Desc: image002.jpg URL: From Rick.Knoblaugh at lsi.com Sun Nov 24 15:27:13 2013 From: Rick.Knoblaugh at lsi.com (Knoblaugh, Rick) Date: Sun, 24 Nov 2013 23:27:13 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> Message-ID: <18a4d76d1d7d4246837e015178ec5a54@DM2PR07MB285.namprd07.prod.outlook.com> Hi Yong, One other minor note, please put the #define DumpBufferSize in uppercase. Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Judy Brock-SSI Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ‘1’. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn’t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I’d very appreciate it. Holiday is upon us and we’d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: ・ nvmeInit.c changes to manage buffers and IO queues in dump modes. ・ nvmeIo.c minor tweak in dump mode where only boot-CPU is available ・ nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). ・ nvmeStat.c helper function change due to some timer related API not available in dump mode. ・ nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From judy.brock at ssi.samsung.com Sun Nov 24 19:36:52 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Mon, 25 Nov 2013 03:36:52 +0000 Subject: [nvmewin] hot plug support Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46C466@SSIEXCH-MB3.ssi.samsung.com> All, Where is the community with official support for hot plug (surprise removal/surprise add) in the OFA Windows driver? Thanks, Judy -------------- next part -------------- An HTML attachment was scrubbed... URL: From Yong.sc.Chen at huawei.com Sun Nov 24 21:31:32 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Mon, 25 Nov 2013 05:31:32 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <18a4d76d1d7d4246837e015178ec5a54@DM2PR07MB285.namprd07.prod.outlook.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com>, <18a4d76d1d7d4246837e015178ec5a54@DM2PR07MB285.namprd07.prod.outlook.com> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE5362C@sjceml501-mbs.china.huawei.com> Thanks, Rick, Will do. Yong Sent from my Windows Phone ________________________________ From: Knoblaugh, Rick Sent: 11/24/2013 3:27 PM To: Judy Brock-SSI; Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, One other minor note, please put the #define DumpBufferSize in uppercase. Thanks, -Rick From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Judy Brock-SSI Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ‘1’. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn’t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I’d very appreciate it. Holiday is upon us and we’d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: ・ nvmeInit.c changes to manage buffers and IO queues in dump modes. ・ nvmeIo.c minor tweak in dump mode where only boot-CPU is available ・ nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). ・ nvmeStat.c helper function change due to some timer related API not available in dump mode. ・ nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From Yong.sc.Chen at huawei.com Mon Nov 25 10:53:18 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Mon, 25 Nov 2013 18:53:18 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ‘1’. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn’t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I’d very appreciate it. Holiday is upon us and we’d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: ・ nvmeInit.c changes to manage buffers and IO queues in dump modes. ・ nvmeIo.c minor tweak in dump mode where only boot-CPU is available ・ nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). ・ nvmeStat.c helper function change due to some timer related API not available in dump mode. ・ nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok’s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,… Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do “!process -1 f” 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can’t do hibernation (no S4). ________________________________ Yong Chen Storage Architect 华为技术有限公司 Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com 本邮件及其附件含有华为公司的保密信息，仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、或散发）本邮件中 的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本邮件！ This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From judy.brock at ssi.samsung.com Mon Nov 25 14:18:15 2013 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Mon, 25 Nov 2013 22:18:15 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> Hi Yong, Ensuring CSTS.RDY == 0 before setting CC.EN to ��1�� is required by the spec �C it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is a sanity check that needs to take place immediately before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion. I don��t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn��t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned. Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself. As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don��t enhance your current changes one way or the other. I say, put the fix in �C it��s fairly trivial. Thanks, Judy From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 10:53 AM To: Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ��1��. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn��t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I��d very appreciate it. Holiday is upon us and we��d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: �� nvmeInit.c changes to manage buffers and IO queues in dump modes. �� nvmeIo.c minor tweak in dump mode where only boot-CPU is available �� nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). �� nvmeStat.c helper function change due to some timer related API not available in dump mode. �� nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok��s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,�� Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do ��!process -1 f�� 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can��t do hibernation (no S4). ________________________________ Yong Chen Storage Architect ��Ϊ�������޹�˾ Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com ���ʼ����丽�����л�Ϊ��˾�ı�����Ϣ�������ڷ��͸������ַ���г��ĸ��˻�Ⱥ�顣�� ֹ�κ����������κ���ʽʹ�ã�������������ȫ���򲿷ֵ�й¶�����ơ���ɢ�������ʼ��� ����Ϣ������������˱��ʼ������������绰���ʼ�֪ͨ�����˲�ɾ�����ʼ��� This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 6737 bytes Desc: image002.jpg URL: From carolyn.d.foster at intel.com Mon Nov 25 15:25:17 2013 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Mon, 25 Nov 2013 23:25:17 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Hi Yong, I also have some feedback and questions about the code changes. 1. I am surprised that there are no ntldrDump checks around the DPC initialization and calls. I wouldn��t have expected the DPCs to work properly on the Windows 7 systems. 2. In function NVMeAllocateMem, the ntldrDump check is wrapped such that if no buffer is allocated from the DumpBuffer, the code path could end up calling StorPortAllocateContiguousMemory in ntldrDump mode. Will this cause a BSOD? Or will it just fail? 3. In RecoveryDpcRoutine new code has been added above the reset adapter call not related to ntldrDump. If the controller isn��t responding, this additional delay time could cause a DPC watchdog timeout bugcheck if the maximum time allowed for a DPC is exceeded. I have some concerns about this new code, what was your reasoning for adding it? Thanks, Carolyn From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Judy Brock-SSI Sent: Monday, November 25, 2013 3:18 PM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi Yong, Ensuring CSTS.RDY == 0 before setting CC.EN to ��1�� is required by the spec �C it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is a sanity check that needs to take place immediately before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion. I don��t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn��t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned. Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself. As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don��t enhance your current changes one way or the other. I say, put the fix in �C it��s fairly trivial. Thanks, Judy From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 10:53 AM To: Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ��1��. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn��t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I��d very appreciate it. Holiday is upon us and we��d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: �� nvmeInit.c changes to manage buffers and IO queues in dump modes. �� nvmeIo.c minor tweak in dump mode where only boot-CPU is available �� nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). �� nvmeStat.c helper function change due to some timer related API not available in dump mode. �� nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok��s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,�� Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do ��!process -1 f�� 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can��t do hibernation (no S4). ________________________________ Yong Chen Storage Architect ��Ϊ�������޹�˾ Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com ���ʼ����丽�����л�Ϊ��˾�ı�����Ϣ�������ڷ��͸������ַ���г��ĸ��˻�Ⱥ�顣�� ֹ�κ����������κ���ʽʹ�ã�������������ȫ���򲿷ֵ�й¶�����ơ���ɢ�������ʼ��� ����Ϣ������������˱��ʼ������������绰���ʼ�֪ͨ�����˲�ɾ�����ʼ��� This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From Yong.sc.Chen at huawei.com Mon Nov 25 22:23:01 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Tue, 26 Nov 2013 06:23:01 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE53AA3@sjceml501-mbs.china.huawei.com> Hi, Carolyn, Thanks for the input. 1. There is no DPC or interrupt in dump mode. It is the polling method to call the same routine. Could you elaborate why you don��t expect DPCs to work properly on Win7? 2. The small buffer is reserved during normal driver load. In dump mode you can��t allocate anything (doc says paltry 64KB). That dump buffer is guaranteed when entering dump mode. Regardless, the same code path would fail not BSOD if BUFFER is NULL as in any other cases. 3. I added the code because for a long time I was and still am dealing with engineer-sample hardware not finished products. After several revisions, they are much more mature now than earlier this year. This is how I make them in consistent ready mode. The rationale for cycling of CC.EN bit during resuming from hibernation is just to mirror normal Initialization step. The timeout is predefined value STORPORT_TIMER_CB_us. For mal-function hardware, the same logic would already have experienced same problems in NVMeInitialize() at raised level. The best way to decide is to test on different implementation of NVMe devices from various vendors and see whether we need to tune these values. Hope these help, Thanks, Yong From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Monday, November 25, 2013 3:25 PM To: Judy Brock-SSI; Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I also have some feedback and questions about the code changes. 1. I am surprised that there are no ntldrDump checks around the DPC initialization and calls. I wouldn��t have expected the DPCs to work properly on the Windows 7 systems. 2. In function NVMeAllocateMem, the ntldrDump check is wrapped such that if no buffer is allocated from the DumpBuffer, the code path could end up calling StorPortAllocateContiguousMemory in ntldrDump mode. Will this cause a BSOD? Or will it just fail? 3. In RecoveryDpcRoutine new code has been added above the reset adapter call not related to ntldrDump. If the controller isn��t responding, this additional delay time could cause a DPC watchdog timeout bugcheck if the maximum time allowed for a DPC is exceeded. I have some concerns about this new code, what was your reasoning for adding it? Thanks, Carolyn From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Judy Brock-SSI Sent: Monday, November 25, 2013 3:18 PM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi Yong, Ensuring CSTS.RDY == 0 before setting CC.EN to ��1�� is required by the spec �C it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is a sanity check that needs to take place immediately before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion. I don��t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn��t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned. Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself. As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don��t enhance your current changes one way or the other. I say, put the fix in �C it��s fairly trivial. Thanks, Judy From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 10:53 AM To: Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ��1��. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn��t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I��d very appreciate it. Holiday is upon us and we��d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: �� nvmeInit.c changes to manage buffers and IO queues in dump modes. �� nvmeIo.c minor tweak in dump mode where only boot-CPU is available �� nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). �� nvmeStat.c helper function change due to some timer related API not available in dump mode. �� nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok��s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,�� Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do ��!process -1 f�� 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can��t do hibernation (no S4). ________________________________ Yong Chen Storage Architect ��Ϊ�������޹�˾ Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com ���ʼ����丽�����л�Ϊ��˾�ı�����Ϣ�������ڷ��͸������ַ���г��ĸ��˻�Ⱥ�顣�� ֹ�κ����������κ���ʽʹ�ã�������������ȫ���򲿷ֵ�й¶�����ơ���ɢ�������ʼ��� ����Ϣ������������˱��ʼ������������绰���ʼ�֪ͨ�����˲�ɾ�����ʼ��� This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 6737 bytes Desc: image002.jpg URL: From Yong.sc.Chen at huawei.com Mon Nov 25 22:28:27 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Tue, 26 Nov 2013 06:28:27 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE53AB8@sjceml501-mbs.china.huawei.com> As trivial as they are, I prefer to deal with cleaning-up changes in early phases in each release cycle, not bundle with complex changes at time trying to ship a bi-annual release. In 1.4 we will have plenty time to focus on performance, stability etc. Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Monday, November 25, 2013 2:18 PM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Ensuring CSTS.RDY == 0 before setting CC.EN to ��1�� is required by the spec �C it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is a sanity check that needs to take place immediately before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion. I don��t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn��t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned. Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself. As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don��t enhance your current changes one way or the other. I say, put the fix in �C it��s fairly trivial. Thanks, Judy From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 10:53 AM To: Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ��1��. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn��t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I��d very appreciate it. Holiday is upon us and we��d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: �� nvmeInit.c changes to manage buffers and IO queues in dump modes. �� nvmeIo.c minor tweak in dump mode where only boot-CPU is available �� nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). �� nvmeStat.c helper function change due to some timer related API not available in dump mode. �� nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok��s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,�� Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do ��!process -1 f�� 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can��t do hibernation (no S4). ________________________________ Yong Chen Storage Architect ��Ϊ�������޹�˾ Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com ���ʼ����丽�����л�Ϊ��˾�ı�����Ϣ�������ڷ��͸������ַ���г��ĸ��˻�Ⱥ�顣�� ֹ�κ����������κ���ʽʹ�ã�������������ȫ���򲿷ֵ�й¶�����ơ���ɢ�������ʼ��� ����Ϣ������������˱��ʼ������������绰���ʼ�֪ͨ�����˲�ɾ�����ʼ��� This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From carolyn.d.foster at intel.com Tue Nov 26 12:08:44 2013 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Tue, 26 Nov 2013 20:08:44 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: <02EC085151D99A469E06988E94FEBCDB1CE53AA3@sjceml501-mbs.china.huawei.com> References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE53AA3@sjceml501-mbs.china.huawei.com> Message-ID: See my comments below in green. Also I found two more issues: According to MSDN StorportEnablePassiveInitialization will fail if the system does not support DPCs, which is the case in crashdump mode. We should have a check for ntldrDump in NVMeInitialize and call NVMePassiveInitialize directly if it��s true. The other potential issue is that in NVMeCallArbiter, in ntldrDump mode we will call NVMeRunning. This could cause a very deep call stack since NVMeCallArbiter is also called from NVMeRunning. In hibernate mode we have limited memory and this could cause issues. I suggest making modifications to NVMeRunningStartAttempt around the NVMeRunning call. It could be a while-loop that would call NVMeRunning if ntldrDump is true, with a stall execution, that would loop until the nextDriverState is start complete, or failed. Carolyn From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 11:23 PM To: Foster, Carolyn D; Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Carolyn, Thanks for the input. 1. There is no DPC or interrupt in dump mode. It is the polling method to call the same routine. Could you elaborate why you don��t expect DPCs to work properly on Win7? I couldn��t find where polling mode is set in the hibernate path, and I was also expecting to see the DPC initialization calls wrapped in ntldrDump checks. Specifically in NVMePassiveInitialize. 2. The small buffer is reserved during normal driver load. In dump mode you can��t allocate anything (doc says paltry 64KB). That dump buffer is guaranteed when entering dump mode. Regardless, the same code path would fail not BSOD if BUFFER is NULL as in any other cases. I��m specifically asking about line 189 in nvmeInit. It seems possible for that line of code, the StorportAllocateContiguousMemory to be called in the crashdump path. Can you confirm that function call in crashdump/hibernate mode simply fails? If it does then I agree that a null buffer here will likely not crash the system. 3. I added the code because for a long time I was and still am dealing with engineer-sample hardware not finished products. After several revisions, they are much more mature now than earlier this year. This is how I make them in consistent ready mode. The rationale for cycling of CC.EN bit during resuming from hibernation is just to mirror normal Initialization step. The timeout is predefined value STORPORT_TIMER_CB_us. For mal-function hardware, the same logic would already have experienced same problems in NVMeInitialize() at raised level. The best way to decide is to test on different implementation of NVMe devices from various vendors and see whether we need to tune these values. My concern here is that the RecoveryDpc routine is not just called during hibernate, it is called during runtime if windows needs to reset the controller. I��m concerned with how these changes impact the normal runtime driver. Did you test this function during runtime? What happens if the maximum time is spent in it? Hope these help, Thanks, Yong From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Monday, November 25, 2013 3:25 PM To: Judy Brock-SSI; Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I also have some feedback and questions about the code changes. 1. I am surprised that there are no ntldrDump checks around the DPC initialization and calls. I wouldn��t have expected the DPCs to work properly on the Windows 7 systems. 2. In function NVMeAllocateMem, the ntldrDump check is wrapped such that if no buffer is allocated from the DumpBuffer, the code path could end up calling StorPortAllocateContiguousMemory in ntldrDump mode. Will this cause a BSOD? Or will it just fail? 3. In RecoveryDpcRoutine new code has been added above the reset adapter call not related to ntldrDump. If the controller isn��t responding, this additional delay time could cause a DPC watchdog timeout bugcheck if the maximum time allowed for a DPC is exceeded. I have some concerns about this new code, what was your reasoning for adding it? Thanks, Carolyn From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Judy Brock-SSI Sent: Monday, November 25, 2013 3:18 PM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi Yong, Ensuring CSTS.RDY == 0 before setting CC.EN to ��1�� is required by the spec �C it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is a sanity check that needs to take place immediately before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion. I don��t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn��t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned. Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself. As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don��t enhance your current changes one way or the other. I say, put the fix in �C it��s fairly trivial. Thanks, Judy From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 10:53 AM To: Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ��1��. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn��t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I��d very appreciate it. Holiday is upon us and we��d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: �� nvmeInit.c changes to manage buffers and IO queues in dump modes. �� nvmeIo.c minor tweak in dump mode where only boot-CPU is available �� nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). �� nvmeStat.c helper function change due to some timer related API not available in dump mode. �� nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok��s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,�� Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do ��!process -1 f�� 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can��t do hibernation (no S4). ________________________________ Yong Chen Storage Architect ��Ϊ�������޹�˾ Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com ���ʼ����丽�����л�Ϊ��˾�ı�����Ϣ�������ڷ��͸������ַ���г��ĸ��˻�Ⱥ�顣�� ֹ�κ����������κ���ʽʹ�ã�������������ȫ���򲿷ֵ�й¶�����ơ���ɢ�������ʼ��� ����Ϣ������������˱��ʼ������������绰���ʼ�֪ͨ�����˲�ɾ�����ʼ��� This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 6737 bytes Desc: image001.jpg URL: From Yong.sc.Chen at huawei.com Tue Nov 26 14:13:10 2013 From: Yong.sc.Chen at huawei.com (Yong Chen) Date: Tue, 26 Nov 2013 22:13:10 +0000 Subject: [nvmewin] code review: crash dump & hibernation support In-Reply-To: References: <02EC085151D99A469E06988E94FEBCDB1CE52822@sjceml501-mbs.china.huawei.com> <645bf3af.00001a88.00000007@n9228> <02EC085151D99A469E06988E94FEBCDB1CE53382@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C349@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE537B2@sjceml501-mbs.china.huawei.com> <36E8D38D6B771A4BBDB1C0D800158A514E46C70B@SSIEXCH-MB3.ssi.samsung.com> <02EC085151D99A469E06988E94FEBCDB1CE53AA3@sjceml501-mbs.china.huawei.com> Message-ID: <02EC085151D99A469E06988E94FEBCDB1CE53C99@sjceml501-mbs.china.huawei.com> See inline. Thanks, Yong From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Tuesday, November 26, 2013 12:09 PM To: Yong Chen; Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support See my comments below in green. Also I found two more issues: According to MSDN StorportEnablePassiveInitialization will fail if the system does not support DPCs, which is the case in crashdump mode. We should have a check for ntldrDump in NVMeInitialize and call NVMePassiveInitialize directly if it��s true. [Yong>] are you talking about this block? You are right, that is exactly the case. /* * When Crashdump/Hibernation driver is being loaded, need to complete the * entire initialization here. In the case of normal driver loading, enable * passive initialization and let NVMePassiveInitialization handle the rest * of the initialization */ if (pAE->ntldrDump == FALSE) { �� /* Call StorPortPassiveInitialization to enable passive init */ StorPortEnablePassiveInitialization(pAE, NVMePassiveInitialize); The other potential issue is that in NVMeCallArbiter, in ntldrDump mode we will call NVMeRunning. This could cause a very deep call stack since NVMeCallArbiter is also called from NVMeRunning. In hibernate mode we have limited memory and this could cause issues. I suggest making modifications to NVMeRunningStartAttempt around the NVMeRunning call. It could be a while-loop that would call NVMeRunning if ntldrDump is true, with a stall execution, that would loop until the nextDriverState is start complete, or failed. [Yong>] you are right about the call stack. In the while loop we could create a new separate mini state-machine for the dump mode initialization. Given more time I can experiment with it. Carolyn From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 11:23 PM To: Foster, Carolyn D; Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Carolyn, Thanks for the input. 1. There is no DPC or interrupt in dump mode. It is the polling method to call the same routine. Could you elaborate why you don��t expect DPCs to work properly on Win7? I couldn��t find where polling mode is set in the hibernate path, and I was also expecting to see the DPC initialization calls wrapped in ntldrDump checks. Specifically in NVMePassiveInitialize. [Yong>] in dump mode the driver model behaves as if in polling mode. Not something explicit to set. NVMePassiveInitialize (&DPC initialization calls) won��t be issued in dump mode. see first comment. 2. The small buffer is reserved during normal driver load. In dump mode you can��t allocate anything (doc says paltry 64KB). That dump buffer is guaranteed when entering dump mode. Regardless, the same code path would fail not BSOD if BUFFER is NULL as in any other cases. I��m specifically asking about line 189 in nvmeInit. It seems possible for that line of code, the StorportAllocateContiguousMemory to be called in the crashdump path. Can you confirm that function call in crashdump/hibernate mode simply fails? If it does then I agree that a null buffer here will likely not crash the system. [Yong>] I see. This new API call is recently merged change. I have not got into this failure case. From my earlier experience with other allocation function (not this new API), all these APIs simply fail with NULL returned. 3. I added the code because for a long time I was and still am dealing with engineer-sample hardware not finished products. After several revisions, they are much more mature now than earlier this year. This is how I make them in consistent ready mode. The rationale for cycling of CC.EN bit during resuming from hibernation is just to mirror normal Initialization step. The timeout is predefined value STORPORT_TIMER_CB_us. For mal-function hardware, the same logic would already have experienced same problems in NVMeInitialize() at raised level. The best way to decide is to test on different implementation of NVMe devices from various vendors and see whether we need to tune these values. My concern here is that the RecoveryDpc routine is not just called during hibernate, it is called during runtime if windows needs to reset the controller. I��m concerned with how these changes impact the normal runtime driver. Did you test this function during runtime? What happens if the maximum time is spent in it? [Yong>] yes, it will be called by windows if hardware misbehaves, trying to reset the hardware. I didn��t try to simulate this scenario, not easy with real hardware. With this change, this same reset is being exercised every time in resuming from hibernation. we can find out by turning on fault injections, however it is not something we have been running regularly. Hope these help, Thanks, Yong From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Monday, November 25, 2013 3:25 PM To: Judy Brock-SSI; Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I also have some feedback and questions about the code changes. 1. I am surprised that there are no ntldrDump checks around the DPC initialization and calls. I wouldn��t have expected the DPCs to work properly on the Windows 7 systems. 2. In function NVMeAllocateMem, the ntldrDump check is wrapped such that if no buffer is allocated from the DumpBuffer, the code path could end up calling StorPortAllocateContiguousMemory in ntldrDump mode. Will this cause a BSOD? Or will it just fail? 3. In RecoveryDpcRoutine new code has been added above the reset adapter call not related to ntldrDump. If the controller isn��t responding, this additional delay time could cause a DPC watchdog timeout bugcheck if the maximum time allowed for a DPC is exceeded. I have some concerns about this new code, what was your reasoning for adding it? Thanks, Carolyn From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Judy Brock-SSI Sent: Monday, November 25, 2013 3:18 PM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi Yong, Ensuring CSTS.RDY == 0 before setting CC.EN to ��1�� is required by the spec �C it is a necessary part of enabling the adapter. Therefore it is not overloading the function and should be kept together. It is a sanity check that needs to take place immediately before writing CC.EN. They should not be separated by other blocks of code. That is not flexibility, it is a design flaw in my opinion. I don��t see how it could possibly result in any destabilization of major features to make sure the RDY bit is not on before setting EN bit in the routine which is dedicated to enabling the controller. If you are worried about removing other checks for CSTS.RDY == 0, then by all means, leave them in. It doesn��t a hurt a thing to have those xtra check points in the two non-runtime paths you mentioned. Conversely, it does potentially hurt to not have an explicit check in the NVMeEnableAdapter itself. As I mentioned previously, there is no check in the PassiveInitialization path for CSTS.RDY == 0 before calling NVMeEnableAdapter in the current code; so we are still violating the spec if we don��t enhance your current changes one way or the other. I say, put the fix in �C it��s fairly trivial. Thanks, Judy From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Monday, November 25, 2013 10:53 AM To: Judy Brock-SSI; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi, Judy, Thanks for your input. I agreed with what you trying to achieve. I also think that block of cycling CC.EN 1->0 can be further refactored into one standalone helper function, to be called by RecoveryDpcRoutine() and NVMeInitialize(). Embedding into NVMeEnableAdapter() would make that function overloaded more than its name & meant to do, and losing the flexibility. Plus they are separated by other blocks of code and would materially change the code, currently for no obvious reason. I would try the next guy to the right thing, it is always hard to fix something, if ever happened in the future. Unless the test is completely reset for this check-in, I would delay this further improvement of refactoring to next time, to keep it separate from and avoid destabilizing this major feature work. What do other folks think? Thanks, Yong From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Sunday, November 24, 2013 8:20 AM To: Yong Chen; Alex Chang; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, I suggest you change the function NVMeEnableAdapter to check for CSTS.RDY == 0 before setting CC.EN to ��1��. You added a check for this some lines above the call in NVMeInitialize. But I think we should avoid decoupling the check for CSTS.RDY == 0 from the controller enable itself. If it is not in the same function, it can be overlooked. For exampley, there is another call to NVMeEnableAdapter in PassiveIntialize that doesn��t check before calling. I would modify NVMeEnableAdapter as below (additions/changes in highlight), change the prototype, and have callers check for success or failure: * @return BOOLEAN * TRUE - If Adapter is enabled correctly * FALSE - If anything goes wrong ******************************************************************************/ BOOLEAN NVMeEnableAdapter( PNVME_DEVICE_EXTENSION pAE ) { PQUEUE_INFO pQI = &pAE->QueueInfo; NVMe_CONTROLLER_CONFIGURATION CC = {0}; NVMe_CONTROLLER_STATUS CSTS = {0}; ULONG PollMax = pAE->uSecCrtlTimeout / MAX_STATE_STALL_us; ULONG PollCount; /* * Program Admin queue registers before enabling the adapter: * Admin Queue Attributes */ StorPortWriteRegisterUlong( pAE, (PULONG)(&pAE->pCtrlRegister->AQA), (pQI->pSubQueueInfo->SubQEntries - 1) + ((pQI->pCplQueueInfo->CplQEntries - 1) << NVME_AQA_CQS_LSB)); . . . (further down): StorPortDebugPrint(INFO, "NVMeEnableAdapter: Setting EN...\n"); /* * Set up Controller Configuration Register */ /* After reset, we must wait for CSTS.RDY == 0 before setting CC.EN to 1 */ for (PollCount = 0; PollCount < PollMax; PollCount++) { CSTS.AsUlong = StorPortReadRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CSTS.AsUlong)); if (CSTS.RDY == 0) { /* Move on if RDY bit is cleared */ break; } NVMeStallExecution(pAE, MAX_STATE_STALL_us); } if (CSTS.RDY != 0) { /* If RDY bit won't clear we can't enable the adapter */ return FALSE; } CC.EN = 1; CC.CSS = NVME_CC_NVM_CMD; CC.MPS = (PAGE_SIZE >> NVME_MEM_PAGE_SIZE_SHIFT); CC.AMS = NVME_CC_ROUND_ROBIN; CC.SHN = NVME_CC_SHUTDOWN_NONE; CC.IOSQES = NVME_CC_IOSQES; CC.IOCQES = NVME_CC_IOCQES; StorPortWriteRegisterUlong(pAE, (PULONG)(&pAE->pCtrlRegister->CC), CC.AsUlong); return TRUE; } /* NVMeEnableAdapter */ Thanks, Judy From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Yong Chen Sent: Friday, November 22, 2013 1:48 PM To: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi, everyone, I hope many are busy testing the changes on your devices. If you have any feedback to share, I��d very appreciate it. Holiday is upon us and we��d like to wrap up this much delayed soon. Thanks, Yong From: Yong Chen Sent: Wednesday, November 20, 2013 12:52 AM To: 'Alex Chang'; Uma Parepalli Cc: 'nvmewin at lists.openfabrics.org' Subject: RE: [nvmewin] code review: crash dump & hibernation support Object #1: crash dump when blue screen or manual triggered, for all SKUs (server or client). Object #2: hibernate and then resume on all client SKUs. + Minor cleaning up and fixes along the way. High-level Summary: The major change is to enable ntldrDump mode so that during crash dump or hibernation, the system memory can be dumped to pre-allocated block locations (MEMORY.DMP or HIBERFIL.SYS file). The same nvme.sys driver will be reloaded as another image into strict dumbed-down environment where usual APIs are not available anymore. The next challenge is to re-initialize the controller properly after having resumed from hibernation image and to continue serve as system boot disk. I need to give credits to earlier contributors (Intel, LSI, IDT and others) for having laid solid building blocks needed for the dump mode. This change solved the buffer issue and introduced a different code path for IOs in dump mode. Detailed Briefs: �� nvmeInit.c changes to manage buffers and IO queues in dump modes. �� nvmeIo.c minor tweak in dump mode where only boot-CPU is available �� nvmeSnti.c fix existing bug where FLUSH cmd should include NSID (all NSs in this case). �� nvmeStat.c helper function change due to some timer related API not available in dump mode. �� nvmeStd.c A: refactored code into NVMeWaitForCtrlRDY() for waiting after setting CC.EN = 1. B: introduced same waiting logic after clearing CC.EN = 0. C: during power up, Reset will issue DPC to call RecoveryDpcRoutine() to re-initialize the driver, similarly the above A+B steps are introduced. Using trunk version code, the hardware I have always timed out on initialization. I have had this fix since this spring. I think it is the same issue listed in Kwok��s laundry list. But I would need Judy to verify whether the issue she found is fixed or not by this change. (Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset.) From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, November 19, 2013 5:04 PM To: Uma Parepalli; Yong Chen Subject: RE: [nvmewin] code review: crash dump & hibernation support Hi Yong, Could you please summarize the changes you made? Normally, we list the changes under each file as high-level briefs. Thanks, Alex From: Uma Parepalli [mailto:uma.parepalli at skhms.com] Sent: Tuesday, November 19, 2013 4:05 PM To: Alex Chang Subject: RE: [nvmewin] code review: crash dump & hibernation support Is there a change log file or something that explains what changes are made without looking at the code? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, November 19, 2013 4:05 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] code review: crash dump & hibernation support Hi all, Please find the attached code changes made by Yong Chen from Huawei. Please review the changes, test them accordingly and provide your feedbacks. Thanks, Alex From: Yong Chen [mailto:Yong.sc.Chen at huawei.com] Sent: Tuesday, November 19, 2013 3:59 PM To: nvmewin at lists.openfabrics.org; Alex Chang Subject: RE: code review: crash dump & hibernation support Hi, all, Please download the source code from the link in the attached email (you need to have Silverlight installed). Or to save the trouble for everyone,�� Alex, Could you reply back with the code change you downloaded. Testlog is attached and see below for the list of tests. Thanks, Yong From: Yong Chen Sent: Monday, November 18, 2013 4:13 PM To: 'nvmewin at lists.openfabrics.org'; 'Alex Chang' Hi, Alex and all, Here is the code change to support crash dump and hibernation. Please review. Hopefully we can wrap up by this week before the meeting. Using the trunk version I had problem with the initialization as well. The trunk version would timeout on me. I think it is the same CSTS.RDY issue Judy raised up. I refactored a bit and fixed it at least for the hardware I have. Thanks, Yong Tests that I have gone thru: 1. manual crash dump. KD>.crash and then reboot and KD -z -v MEMORY.DMP, do ��!process -1 f�� 2. manual hibernation or pwrtest /sleep /c:10 /d:30 /p:30 /s:4 3. SCSICompliance (log attached). 4. stresses: iostress, sdstress (log attached) 5. hibernation has been tested on win8.0 as well, but not extensively. 6. Hibernation has also been tested with both bootable OptionROM or the newly released UEFI driver. 7. All tests were conducted on x64 platform, involving 3 different hardware, plus another Intel MB which can��t do hibernation (no S4). ________________________________ Yong Chen Storage Architect ��Ϊ�������޹�˾ Huawei Technologies Co., Ltd [Company_logo]Office: 408-330-5482 Mobile: 425-922-0658 Email: yong.sc.chen at huawei.com 2330 Central Expressway Santa Clara, CA 95050 http://www.huawei.com ���ʼ����丽�����л�Ϊ��˾�ı�����Ϣ�������ڷ��͸������ַ���г��ĸ��˻�Ⱥ�顣�� ֹ�κ����������κ���ʽʹ�ã�������������ȫ���򲿷ֵ�й¶�����ơ���ɢ�������ʼ��� ����Ϣ������������˱��ʼ������������绰���ʼ�֪ͨ�����˲�ɾ�����ʼ��� This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.jpg Type: image/jpeg Size: 6737 bytes Desc: image002.jpg URL: From Kwok.Kong at pmcs.com Tue Nov 26 18:19:35 2013 From: Kwok.Kong at pmcs.com (Kwok Kong) Date: Wed, 27 Nov 2013 02:19:35 +0000 Subject: [nvmewin] Working Group meeting minutes Nov 25, 2013 Message-ID: <03D88B383FA04244AA514AA931F7B1290CBA8E06@BBYEXM01.pmc-sierra.internal> NVMe OFA Windows Driver Meeting Note (Nov 25, 2013) Meeting Status ============== 1. Alex Chang of PMC Sierra has taken over the role of source maintainer from Ray Robles of Intel. Thanks to Ray for his contribution in the last two years and Alex to take over. 2. Release 1.3 is expected to be released by the end of Dec 2013 assuming that we can get volunteers to resolve all known issues. 3. The focus for the releases in 2014 is to focus on stability and not features. Getting WHQL certification is going to be the top objective. 4. The following features will be included in the 1.3 release - Windows 32-bit support (Dharani Kote - Sandisk) - Windows 8 extended SRB support (Alex Chang - PMC) - Hibernation (Yong Chen - Huawei) 5. The following issues will be fixed in the 1.3 release - NUMA group support in core enumeration - (Alex Chang - PMC) - Core-MSI vector queue mapping issues - CMD_ENTRY synchronization issues - Remove using mask bits as core index to allocate core tables - (Alex Chang - PMC) - Paramlist length problem - (Alex Chang - PMC) - NVMeInitAdminQueues return value - (Alex Chang - PMC) - Performance issue in Windows 2012 and Windows 8. - (Alex Chang - PMC) - freeQList Access - (Alex Chang - PMC) 6. There is on owner for the following issues yet. We need volunteers. Please sign up to fix the following problems. - remove #define for CHATHAM2 - Not handling CSTS.RDY status (from 1->0 and 0->1) properly on NVMe reset - Controller reset does not handle all cases - Learning of CPU core to Vector failure handling - BUILDIO - PRP list building problem - orphaned requests 7. 1.3 release will be delayed if we cannot get enough volunteers to fix the issues as stated in item 6. 8. The team will meet again in Jan 2014 to discuss the release plan for 2014. 9. The following features has been deferred to 2014: - End to end protection support - Driver tracing feature - Robert Randall (Micron) - Migrate to VS2013, WDK 8.1 Features that are not supported currently ========================================= NVMe 1.1 support: - multi-path - SGL - Get/Set feature update - Autonomous power state transition - Host Identifier - Reservation Notification Mask - Reservation Persistence - identify structure update - write zeros command Actions ======= 1. Check with intel team to remove #define for CHATHAM2 in the source code - Carolyn Foster (intel) 2. Set up meeting in Jan 2014 to discuss release plan for 2014. - Kwok Kong (PMC) 3. Send up test requirement for submitting patch. - Kwok Kong (PMC) 4. Send out memory/core dump procedure to debug the driver. - Yong Chen (Huawei)