From thomas.freeman at hgst.com Wed Oct 1 08:00:16 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Wed, 1 Oct 2014 15:00:16 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFDD5D.E50FF900] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From thomas.freeman at hgst.com Wed Oct 1 08:31:44 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Wed, 1 Oct 2014 15:31:44 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: <39266611799043979a8a633c171a7923@CO2PR04MB684.namprd04.prod.outlook.com> Hi Alex…. I’m a little confused on this. Here is my understanding of how it works: For this discussion I’ll define 3 values – CQ, SQE and CQS 1. QS: The size of the queue (pSQI->SubQEntries * sizeof(NVMe_COMPLETION_QUEUE_ENTRY) 2. SQE: A pointer to the the first byte after the the submission queue(PtrTemp + queueSize) 3. CQS: a pointer to the Completion Queue (calculated by rounding the SQE to the next page boundary). The actual Completion queue uses memory CQS through (CQS+QS-1) But, the memset in the old code clears memory SQE through (SQE+QS+1) Using this memset does not clear the entire Completion Queue. Memory from (SQE+QS+1) through (CQS+QS+1) is not initialized. The change I made clears memory CQS through (CQS+QS+1). If this is clearing invalid memory, then the Completion Queue itself it using invalid memory. What are your thoughts on this? Thanks, Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, September 30, 2014 10:19 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFDD60.45761840] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Wed Oct 1 08:54:12 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 1 Oct 2014 15:54:12 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <39266611799043979a8a633c171a7923@CO2PR04MB684.namprd04.prod.outlook.com> References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <39266611799043979a8a633c171a7923@CO2PR04MB684.namprd04.prod.outlook.com> Message-ID: Hi Tom, Firstly, like Parag mentioned, we allocated an extra system page for the entries. The QS you defined should be pSQI->SubQEntries * sizeof(NVMe_COMMAND). Please note that the value of queueSize in the codes have be replaced with pSQI->SubQEntries * sizeof(NVMe_COMPLETION_QUEUE_ENTRY) later before memset the CPL entries. Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 8:32 AM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex…. I’m a little confused on this. Here is my understanding of how it works: For this discussion I’ll define 3 values – CQ, SQE and CQS 1. QS: The size of the queue (pSQI->SubQEntries * sizeof(NVMe_COMPLETION_QUEUE_ENTRY) 2. SQE: A pointer to the the first byte after the the submission queue(PtrTemp + queueSize) 3. CQS: a pointer to the Completion Queue (calculated by rounding the SQE to the next page boundary). The actual Completion queue uses memory CQS through (CQS+QS-1) But, the memset in the old code clears memory SQE through (SQE+QS+1) Using this memset does not clear the entire Completion Queue. Memory from (SQE+QS+1) through (CQS+QS+1) is not initialized. The change I made clears memory CQS through (CQS+QS+1). If this is clearing invalid memory, then the Completion Queue itself it using invalid memory. What are your thoughts on this? Thanks, Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, September 30, 2014 10:19 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFDD53.FC6BA1D0] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From parag.sheth at seagate.com Wed Oct 1 11:30:11 2014 From: parag.sheth at seagate.com (Parag Sheth) Date: Wed, 1 Oct 2014 11:30:11 -0700 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> Message-ID: Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman wrote: > Hi Paraq… thanks for the feedback. > > > > I’ll remove the reference to SetWmiDataBlock. > > > > In my notes, I forgot to mention the change I made for mode > SntiHardCodeCacheModePage. With the change from the last patch, SCSI > compliance was failing on Windows 7. (The failure is shown below). Using > PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully > complete that test. I conferred with Alex offline about this fix – he > indicated that he had intended to include this in Patch#27. > > Alex, do you have any comments on this issue? > > > > > > Policy: REQUIRED > > Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., > TUID= > > Description: Checking the Caching Mode Page is 20 bytes. > > Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) > specification Section 6.7 > http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf > > Expectation: cachePageLength == 20 bytes. > > MODE: Cache page length is 0xc. > > > > > > Tom Freeman > > > > Software Engineer, Device Manager and Driver Development > > HGST, a Western Digital company > > Thomas.Freeman at hgst.com > > 507-322-2311 > > > > *From:* Parag Sheth [mailto:parag.sheth at seagate.com] > *Sent:* Tuesday, September 30, 2014 6:07 PM > *To:* Alex Chang > *Cc:* Thomas Freeman; nvmewin at lists.openfabrics.org > *Subject:* Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and > various fixes > > > > Hi Tom, > > > > 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) > overturns the fix made in last patch. > > 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be > removed. > > > > Other than these 2, the patch looks fine. > > > > Thanks > > Parag Sheth > > > > On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang wrote: > > Thank you, Parag. > > I know we allocate one extra page intentionally. When people don’t know > the trick, the original order simply avoids some concern. > > > > Alex > > > > *From:* Thomas Freeman [mailto:thomas.freeman at hgst.com] > *Sent:* Tuesday, September 30, 2014 7:50 AM > *To:* Parag Sheth; Alex Chang > *Cc:* nvmewin at lists.openfabrics.org > *Subject:* RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and > various fixes > > > > Thank you Paraq… I like your explanation. > > In addition to your explanation, I’d also say the drivers uses pCplQStart > -> (pCplQstart + queueSize -1) for the completion queue. > > That is also the address range used in that memset. If those address are > wrong, then the CompletionQueue itself is wrong. > > > > Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I > didn’t use memset to begin with. > > > > Tom Freeman > > > > Software Engineer, Device Manager and Driver Development > > HGST, a Western Digital company > > Thomas.Freeman at hgst.com > > 507-322-2311 > > > > *From:* Parag Sheth [mailto:parag.sheth at seagate.com > ] > *Sent:* Monday, September 29, 2014 8:32 PM > *To:* Alex Chang > *Cc:* Thomas Freeman; nvmewin at lists.openfabrics.org > *Subject:* Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and > various fixes > > > > Hi Alex, > > > > Tom's change related to your point # 2 seems right. > > > > Here is my explanation. > > - While allocating memory for submission queue + completion queue + > command entries, there is 1 additional memory page allocated. > > - Because this allocation is page aligned to begin with, sub queue start > is always page aligned. > > - making completion queue page aligned might use 1 additionally allocated > page. > > - command entries need not be page aligned. > > - And all these 3 things are initialized in the above order. > > > > So I think clearing memory for completion queue must be done only after > page align macro. Let me know if I am missing anything here. > > > > Thanks > > Parag Sheth > > > > On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang wrote: > > Hi Tom, > > > > I have the following comments after browsing the changes you’ve made: > > 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by > memset ? > > 2. In line 1195 of nvmeinit.c, you moved down memset after page > align macro for completion queue start. In case of the original completion > queue start wasn’t page aligned, memset will clear the memory it shouldn’t. > > > > Thanks, > > Alex > > > > *From:* nvmewin-bounces at lists.openfabrics.org [mailto: > nvmewin-bounces at lists.openfabrics.org] *On Behalf Of *Thomas Freeman > *Sent:* Wednesday, September 24, 2014 11:55 AM > *To:* nvmewin at lists.openfabrics.org > *Subject:* [nvmewin] HGST Changes to Enable WMI, Driver Trace and various > fixes > > > > Content-Type: text/plain; charset=UTF-8 > > Content-Transfer-Encoding: 8bit > > Date: %%SENT_DATE%% > > Subject: Suspect Message Quarantined > > > > > > > > WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: > > > > %%DESC%% > > > > The full message and the attachment have been stored in the quarantine. > > > > The identifier for this message is '%%QID%%'. > > > > Access the quarantine at: > > https://puremessage.pmc-sierra.bc.ca:28443/ > > > > For more information on PMC's Anti-Spam system: > > http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ > > > > IT Services > > PureMessage Admin > > > > The attached zip file contains source code, project files and > instructional documents for WMI and Driver tracing. > > The password is “hgst1234” > > > > We tested the changes on w7, w8 and w8.1 – 64-bit > > Testing included sdstress, iometer, quick and slow format: MBR and GPT and > scsi compliance. > > > > > > Here is a description of the various changes: > > > > *WMI *(WindowsWmiDesc.pdf in the attached zip file gives details on using > the change) > > nvme.vcxproj - Menu for "Mof Compiler" > > -Create Binary Mof File = "../../source/%(FileName).bmf" > > -WMI Syntax Check = Yes (-WMI) > > menu for Linker->Input > > -added $(DDK_LIB_PATH)\scsiwmi.lib > > directly in the file. This directive causes VS to create nvmeMofData.h: > > > > > > "$(IntDir)nvmeMofData.h" > > -u -h"$(IntDir)nvmeMofData.h" > > > > > > > > > > nvme.rc - Added "mofResource" statement > > nvmeMofData.bmf - Visual studio generates this file on every compile. > Exists in the "source" directory. > > nvmeMofData.mof – Used to define WMI obects. Exists in the "source" > directory > > nvmeMofData.h - Visual studio generates during compile. It is created > based on nvmeMofData.mof. It exists in the project's intermediate directory > (e.g. nvme/x64/Win7Debug). > > > > nvmeStd.c > > -NVMeFindAdapter - indicate that the driver is a > wmiProvider and initialize the wmiContext. > > -NVMeBuildIo - Allow calls to WMI > > -NVMeStartIo - Allow calls to WMI > > > > nvmeStd.h > > -WmiLibContext - Added this structure to the device > extension. The WmiLibContext structure provides registration information > for a miniport driver's data and event blocks and defines entry points for > the miniport driver's HwScsiWmiXxx callback routines. > > -WmiReqContext - Added this structure to the srb > extension. When processing a WMI command, this structure is used to save > context information for that particular command. > > > > precomp.h > > -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, > nmeWmi.h > > > > > > *WPP - Driver tracing *(WindowsTraceCollection.pdf in the attached zip > file gives details on using the change) > > Relied heavily on the document "Using Storage Tracing Support In Miniport > Drivers" - > http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc > > > > nvme.vcxproj - in the Menu for "Wpp Tracing" > > -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) > > -Run Wpp Tracing = Yes > > -Scan Configuration Data = ..\..\source\nvme_tracing.h > > -Specify Template File = {km-StorDefault.tpl}*.tmh > > -Trace Kernel-Mode Components = Yes (-km) > > > > All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated > file to support driver tracing. > > > > nvmeStd.c - create a global variable for storing the trace context > information. At cleanup time, the Port Driver calls a cleanup routine that > uses this global variable. > > -DriverEntry - Include code to initialize driver tracing > through wpp. > > -WppCleanupRoutine - Added code to be called by the port > driver to clean up driver tracing. > > nvmeStd.h - > > -add function prototype for Wpp Cleanup routine. > > -Changed values for debug print levels to match values in > evntrace.h > > INFO = TRACE_LEVEL_INFORMATION, > > WARNING = TRACE_LEVEL_WARNING, > > ERROR = TRACE_LEVEL_ERROR, > > TRACE = TRACE_LEVEL_VERBOSE > > > > precomp.h > > -added nvme_tracing.h > > > > > > *Various Fixes* > > nvme.vcxproj > > -Changed c/c++ compiler warning level to W3/WX > > nvmeSnti.c > > SntiTranslateReadCapacity16 - The caller’s response buffer was being > cleared and set up without regard to its size. I put in changes to allocate > a read capacity buffer on the stack. Once that temporary buffer was filled > in, I used the response buffer size to determine how much data to copy into > the caller’s response buffer. > > NOTE: It appears this same issue occurs in the handling of other scsi > commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response > buffer was too short, setting the fields in pIdDescriptor would overrun > the response buffer). This may be a fix to be implemented in the future. > The issue in SntiTranslateReadCapacity16 issue was seen during testing. The > potential issue with SntiTranslateDeviceIdentificationPage has not > occurred in our testing. > > > > > > SnitTranslateUnmap > > -Changes to ensure the DataSetManagement buffer was always > 16-byte aligned. > > -Use prp2 to allow the prplist to cross a page boundary > > -When building the dsm-deallocate command, cdw10->NR was > used to count the number of ranges - starting with the value 1. Since NR is > a zero-based value, the count was decremented when command building was > completed. Unfortunately, when 256 ranges are specified, NR has a value of > 0 before decrementing. The value of 0 prevented the code from decrementing > NR. A change was made to use a local uint16 variable to cound the number of > ranges. > > SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based > value. A change was made to correctly set this as a zero-based value. > > SntiTranslateLogSense - A pointer to the data buffer is saved in the > srb extension. A change was made to also save the data buffer length in the > srb extension. The value is used when the data buffer is freed. > > > > nvmeStd.h > > -Increase size of dsmBuffer to allow for 16-byte alignment. > > -add dataBufferSize to store size of data buffer allocated for log > sense command. > > > > > > > > Tom Freeman > > > > Software Engineer, Device Manager and Driver Development > > HGST, a Western Digital company > > Thomas.Freeman at hgst.com > > 507-322-2311 > > > > > _______________________________________________ > nvmewin mailing list > nvmewin at lists.openfabrics.org > http://lists.openfabrics.org/mailman/listinfo/nvmewin > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: not available URL: From Alex.Chang at pmcs.com Wed Oct 1 12:17:23 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 1 Oct 2014 19:17:23 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> Message-ID: Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFDD71.92F7CCB0] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From thomas.freeman at hgst.com Wed Oct 1 12:30:24 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Wed, 1 Oct 2014 19:30:24 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> Message-ID: <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFDD84.3BD1A3D0] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Mon Oct 6 08:50:42 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Mon, 6 Oct 2014 15:50:42 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE142.81CAB3B0] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From thomas.freeman at hgst.com Mon Oct 6 13:26:39 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Mon, 6 Oct 2014 20:26:39 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> Hi Alex…. Now that all the comments are in, I’ll merge in the changes from the review and run a test. I should have the new patch out in the next day. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 6, 2014 10:51 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE179.EB7E0230] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Mon Oct 6 13:28:32 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Mon, 6 Oct 2014 20:28:32 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Really appreciated ! Tom. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Monday, October 06, 2014 1:27 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex…. Now that all the comments are in, I’ll merge in the changes from the review and run a test. I should have the new patch out in the next day. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 6, 2014 10:51 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE169.51C64E00] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Tue Oct 7 09:12:26 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Tue, 7 Oct 2014 16:12:26 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Hi Tom, I have a quick question about the data type of “Id” in class NVMe_DataType defined in nvmeMofData.mof. Is there any specific reason you defined it as “sint32” instead of “uint32”? Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Monday, October 06, 2014 1:27 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex…. Now that all the comments are in, I’ll merge in the changes from the review and run a test. I should have the new patch out in the next day. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 6, 2014 10:51 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE20E.B4F1F8D0] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From thomas.freeman at hgst.com Tue Oct 7 10:17:22 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Tue, 7 Oct 2014 17:17:22 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Here is the revised code code for the WMI/WPP + various changes. The password is "hgst1234" Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is "hgst1234" We tested the changes on w7, w8 and w8.1 - 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof - Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller's response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller's response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1117 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: WMIWppChanges-Revision2.zip Type: application/x-zip-compressed Size: 2012564 bytes Desc: WMIWppChanges-Revision2.zip URL: From thomas.freeman at hgst.com Tue Oct 7 10:36:21 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Tue, 7 Oct 2014 17:36:21 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Alex, That was part of an example I grabbed. But, I’m not aware of any reason it couldn’t be uint32. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 7, 2014 11:12 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, I have a quick question about the data type of “Id” in class NVMe_DataType defined in nvmeMofData.mof. Is there any specific reason you defined it as “sint32” instead of “uint32”? Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Monday, October 06, 2014 1:27 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex…. Now that all the comments are in, I’ll merge in the changes from the review and run a test. I should have the new patch out in the next day. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 6, 2014 10:51 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE22B.4BB3CF20] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From mikeb at bustrace.com Tue Oct 7 12:22:22 2014 From: mikeb at bustrace.com (Mike Berhan) Date: Tue, 7 Oct 2014 13:22:22 -0600 Subject: [nvmewin] HwStorPassiveInitializeRoutine and StorPortNotification->RequestTimerCall Message-ID: <009401cfe264$0564e8e0$102ebaa0$@bustrace.com> I've performed a detailed analysis of the NVMe initialization sequence, with the OFA driver, and posted a video here: http://www.bustrace.com/bustrace10/videos/NVMeBootCapture/ The NVMe passive initialization phase uses a background state machine to initialize the NVMe controller after the controller has been enabled. On my two NUMA node system, with 16 logical processors, the passive initialization phase takes ~1.859 seconds. However, if I analyze the time spent communicating with the NVMe controller, plus the time spent waiting for the NVMe controller to process the request, I come up with less than 170 milliseconds. The bulk of that 1.859 seconds is spent with an idle state machine waiting for the next timer callback, and an idle passive initialization phase waiting for the state machine to complete. One of the primary reasons for this is the use of StorPortNotification - RequestTimerCall. The miniport is using the value of STORPORT_TIMER_CB_us for the delay which equals: #define STORPORT_TIMER_CB_us 5000 /* .005 seconds */ Even though the miniport driver is requesting 5 msec, the actual time to receive the callback is typically 15 msec. The reason for this is noted in the STORPORT documentation: The system timer resolution is approximately 10 milliseconds. This may not be a significant issue but I know some OEMs have concerns about time to ready so I thought I would post my analysis. Mike Berhan busTRACE Technologies ps: If you notice the "Not in memory" for the PERF_CONFIGURATION_DATA structure, that is caused by the miniport driver not setting the Version and Size fields of the data structure. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Tue Oct 7 12:53:48 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Tue, 7 Oct 2014 19:53:48 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: I see. I am sure it’s better defined as uint32 and I will add the change to my patch if you don’t mind. Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, October 07, 2014 10:36 AM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Alex, That was part of an example I grabbed. But, I’m not aware of any reason it couldn’t be uint32. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 7, 2014 11:12 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, I have a quick question about the data type of “Id” in class NVMe_DataType defined in nvmeMofData.mof. Is there any specific reason you defined it as “sint32” instead of “uint32”? Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Monday, October 06, 2014 1:27 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex…. Now that all the comments are in, I’ll merge in the changes from the review and run a test. I should have the new patch out in the next day. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 6, 2014 10:51 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE22D.A13CDD40] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Tue Oct 7 12:55:47 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Tue, 7 Oct 2014 19:55:47 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Dear all, Please review/test the revised patch at your earliest convenience. I will start to collect approvals on Friday. Thank you so much, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Tuesday, October 07, 2014 10:17 AM To: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Here is the revised code code for the WMI/WPP + various changes. The password is "hgst1234" Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is "hgst1234" We tested the changes on w7, w8 and w8.1 - 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof - Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller's response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller's response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From thomas.freeman at hgst.com Tue Oct 7 13:13:42 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Tue, 7 Oct 2014 20:13:42 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <2850_1411584948_542313B4_2850_3247_1_932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <65aee2cb592141bbb4b928be32f870f2@CO2PR04MB684.namprd04.prod.outlook.com> <140822b6a8d2453c8b32a752aaa1e27b@BLUPR04MB673.namprd04.prod.outlook.com> <2ea2cd717d1e47709ab4001538e41614@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: No problem, feel free to make the modification Thanks, Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 7, 2014 2:54 PM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes I see. I am sure it’s better defined as uint32 and I will add the change to my patch if you don’t mind. Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, October 07, 2014 10:36 AM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Alex, That was part of an example I grabbed. But, I’m not aware of any reason it couldn’t be uint32. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Tuesday, October 7, 2014 11:12 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, I have a quick question about the data type of “Id” in class NVMe_DataType defined in nvmeMofData.mof. Is there any specific reason you defined it as “sint32” instead of “uint32”? Thanks, Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Monday, October 06, 2014 1:27 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex…. Now that all the comments are in, I’ll merge in the changes from the review and run a test. I should have the new patch out in the next day. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 6, 2014 10:51 AM To: Thomas Freeman; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Good morning, Tom, Will you please send out the revised patch soon? We still have three patches to go before our November release. Thank you very much! Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Wednesday, October 01, 2014 12:30 PM To: Alex Chang; Parag Sheth Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thanks for the clarification. I’ll change to using CACHING_MOE_PAGE. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Wednesday, October 1, 2014 2:17 PM To: Parag Sheth; Thomas Freeman Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, If we don’t support VS 2012 anymore, the we certainly can use PMODE_CACHING_PAGE_EX for all windows with VS 2013. Because the storport.h included in VS 2012 doesn’t define PMODE_CACHING_PAGE_EX. The safest bet to fix it, I agree to use CACHING_MODE_PAGE defined in nvmesnti.h. Sorry Tom, I did not exam it carefully when we discussed the issue. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Wednesday, October 01, 2014 11:30 AM To: Thomas Freeman Cc: Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, The real solution to this problem is to use CACHING_MODE_PAGE defined in nvmesnti.h and not to use MODE_CACHING_PAGE defined in storport.h. The names are confusing. Thanks Parag Sheth On Wed, Oct 1, 2014 at 8:00 AM, Thomas Freeman > wrote: Hi Paraq… thanks for the feedback. I’ll remove the reference to SetWmiDataBlock. In my notes, I forgot to mention the change I made for mode SntiHardCodeCacheModePage. With the change from the last patch, SCSI compliance was failing on Windows 7. (The failure is shown below). Using PMODE_CACHING_PAGE_EX for all Windows versions allowed us to successfully complete that test. I conferred with Alex offline about this fix – he indicated that he had intended to include this in Patch#27. Alex, do you have any comments on this issue? Policy: REQUIRED Start: MODE SELECT 6: MODE SENSE (6) Checking Caching Mode Page Length., TUID= Description: Checking the Caching Mode Page is 20 bytes. Reference: SCSI Primary Commands - 3 (SPC-3) Revision 23 (or published) specification Section 6.7 http://www.t10.org/ftp/t10/drafts/spc3/spc3r23.pdf Expectation: cachePageLength == 20 bytes. MODE: Cache page length is 0xc. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Tuesday, September 30, 2014 6:07 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Tom, 1. Your change in function SntiHardCodeCacheModePage (nvmesnti.c) overturns the fix made in last patch. 2. Function SetWmiDataBlock (nvmeWmi.c & .h) is never used. It can be removed. Other than these 2, the patch looks fine. Thanks Parag Sheth On Tue, Sep 30, 2014 at 8:18 AM, Alex Chang > wrote: Thank you, Parag. I know we allocate one extra page intentionally. When people don’t know the trick, the original order simply avoids some concern. Alex From: Thomas Freeman [mailto:thomas.freeman at hgst.com] Sent: Tuesday, September 30, 2014 7:50 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Thank you Paraq… I like your explanation. In addition to your explanation, I’d also say the drivers uses pCplQStart -> (pCplQstart + queueSize -1) for the completion queue. That is also the address range used in that memset. If those address are wrong, then the CompletionQueue itself is wrong. Alex…. Yes, I’ll change the RtlZeroMemory to memset. I’m not sure why I didn’t use memset to begin with. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Monday, September 29, 2014 8:32 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, Tom's change related to your point # 2 seems right. Here is my explanation. - While allocating memory for submission queue + completion queue + command entries, there is 1 additional memory page allocated. - Because this allocation is page aligned to begin with, sub queue start is always page aligned. - making completion queue page aligned might use 1 additionally allocated page. - command entries need not be page aligned. - And all these 3 things are initialized in the above order. So I think clearing memory for completion queue must be done only after page align macro. Let me know if I am missing anything here. Thanks Parag Sheth On Mon, Sep 29, 2014 at 11:18 AM, Alex Chang > wrote: [cid:image001.gif at 01CFE241.46537560] Hi Tom, I have the following comments after browsing the changes you’ve made: 1. In line 184 of nvmestd.c, can RtlZeroMemory be replaced by memset ? 2. In line 1195 of nvmeinit.c, you moved down memset after page align macro for completion queue start. In case of the original completion queue start wasn’t page aligned, memset will clear the memory it shouldn’t. Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 11:55 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Thu Oct 9 18:00:31 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 10 Oct 2014 01:00:31 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Hi Carolyn and Parag, Please let me know when you approve the patch. Thanks a lot, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Tuesday, October 07, 2014 10:17 AM To: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Here is the revised code code for the WMI/WPP + various changes. The password is "hgst1234" Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is "hgst1234" We tested the changes on w7, w8 and w8.1 - 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof - Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller's response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller's response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From parag.sheth at seagate.com Fri Oct 10 16:27:28 2014 From: parag.sheth at seagate.com (Parag Sheth) Date: Fri, 10 Oct 2014 16:27:28 -0700 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Hi Alex, The changes look good. Thanks Parag Sheth On Thu, Oct 9, 2014 at 6:00 PM, Alex Chang wrote: > Hi Carolyn and Parag, > > > > Please let me know when you approve the patch. > > > > Thanks a lot, > > Alex > > > > *From:* nvmewin-bounces at lists.openfabrics.org [mailto: > nvmewin-bounces at lists.openfabrics.org] *On Behalf Of *Thomas Freeman > *Sent:* Tuesday, October 07, 2014 10:17 AM > *To:* Thomas Freeman; nvmewin at lists.openfabrics.org > *Subject:* Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and > various fixes > > > > Content-Type: text/plain; charset=UTF-8 > > Content-Transfer-Encoding: 8bit > > Date: %%SENT_DATE%% > > Subject: Suspect Message Quarantined > > > > > > > > WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: > > > > %%DESC%% > > > > The full message and the attachment have been stored in the quarantine. > > > > The identifier for this message is '%%QID%%'. > > > > Access the quarantine at: > > https://puremessage.pmc-sierra.bc.ca:28443/ > > > > For more information on PMC's Anti-Spam system: > > http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ > > > > IT Services > > PureMessage Admin > > > > Here is the revised code code for the WMI/WPP + various changes. > > The password is “hgst1234” > > > > Changes to the original include: > > Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX > > Replacing RtlZeroMemory with memset > > Remove SetWmiDataBlock > > > > > > > > > > Tom Freeman > > > > Software Engineer, Device Manager and Driver Development > > HGST, a Western Digital company > > Thomas.Freeman at hgst.com > > 507-322-2311 > > > > *From:* nvmewin-bounces at lists.openfabrics.org [ > mailto:nvmewin-bounces at lists.openfabrics.org > ] *On Behalf Of *Thomas Freeman > *Sent:* Wednesday, September 24, 2014 1:55 PM > *To:* nvmewin at lists.openfabrics.org > *Subject:* [nvmewin] HGST Changes to Enable WMI, Driver Trace and various > fixes > > > > The attached zip file contains source code, project files and > instructional documents for WMI and Driver tracing. > > The password is “hgst1234” > > > > We tested the changes on w7, w8 and w8.1 – 64-bit > > Testing included sdstress, iometer, quick and slow format: MBR and GPT and > scsi compliance. > > > > > > Here is a description of the various changes: > > > > *WMI *(WindowsWmiDesc.pdf in the attached zip file gives details on using > the change) > > nvme.vcxproj - Menu for "Mof Compiler" > > -Create Binary Mof File = "../../source/%(FileName).bmf" > > -WMI Syntax Check = Yes (-WMI) > > menu for Linker->Input > > -added $(DDK_LIB_PATH)\scsiwmi.lib > > directly in the file. This directive causes VS to create nvmeMofData.h: > > > > > > "$(IntDir)nvmeMofData.h" > > -u -h"$(IntDir)nvmeMofData.h" > > > > > > > > > > nvme.rc - Added "mofResource" statement > > nvmeMofData.bmf - Visual studio generates this file on every compile. > Exists in the "source" directory. > > nvmeMofData.mof – Used to define WMI obects. Exists in the "source" > directory > > nvmeMofData.h - Visual studio generates during compile. It is created > based on nvmeMofData.mof. It exists in the project's intermediate directory > (e.g. nvme/x64/Win7Debug). > > > > nvmeStd.c > > -NVMeFindAdapter - indicate that the driver is a > wmiProvider and initialize the wmiContext. > > -NVMeBuildIo - Allow calls to WMI > > -NVMeStartIo - Allow calls to WMI > > > > nvmeStd.h > > -WmiLibContext - Added this structure to the device > extension. The WmiLibContext structure provides registration information > for a miniport driver's data and event blocks and defines entry points for > the miniport driver's HwScsiWmiXxx callback routines. > > -WmiReqContext - Added this structure to the srb > extension. When processing a WMI command, this structure is used to save > context information for that particular command. > > > > precomp.h > > -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, > nmeWmi.h > > > > > > *WPP - Driver tracing *(WindowsTraceCollection.pdf in the attached zip > file gives details on using the change) > > Relied heavily on the document "Using Storage Tracing Support In Miniport > Drivers" - > http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc > > > > > nvme.vcxproj - in the Menu for "Wpp Tracing" > > -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) > > -Run Wpp Tracing = Yes > > -Scan Configuration Data = ..\..\source\nvme_tracing.h > > -Specify Template File = {km-StorDefault.tpl}*.tmh > > -Trace Kernel-Mode Components = Yes (-km) > > > > All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated > file to support driver tracing. > > > > nvmeStd.c - create a global variable for storing the trace context > information. At cleanup time, the Port Driver calls a cleanup routine that > uses this global variable. > > -DriverEntry - Include code to initialize driver tracing > through wpp. > > -WppCleanupRoutine - Added code to be called by the port > driver to clean up driver tracing. > > nvmeStd.h - > > -add function prototype for Wpp Cleanup routine. > > -Changed values for debug print levels to match values in > evntrace.h > > INFO = TRACE_LEVEL_INFORMATION, > > WARNING = TRACE_LEVEL_WARNING, > > ERROR = TRACE_LEVEL_ERROR, > > TRACE = TRACE_LEVEL_VERBOSE > > > > precomp.h > > -added nvme_tracing.h > > > > > > *Various Fixes* > > nvme.vcxproj > > -Changed c/c++ compiler warning level to W3/WX > > nvmeSnti.c > > SntiTranslateReadCapacity16 - The caller’s response buffer was being > cleared and set up without regard to its size. I put in changes to allocate > a read capacity buffer on the stack. Once that temporary buffer was filled > in, I used the response buffer size to determine how much data to copy into > the caller’s response buffer. > > NOTE: It appears this same issue occurs in the handling of other scsi > commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response > buffer was too short, setting the fields in pIdDescriptor would overrun > the response buffer). This may be a fix to be implemented in the future. > The issue in SntiTranslateReadCapacity16 issue was seen during testing. The > potential issue with SntiTranslateDeviceIdentificationPage has not > occurred in our testing. > > > > > > SnitTranslateUnmap > > -Changes to ensure the DataSetManagement buffer was always > 16-byte aligned. > > -Use prp2 to allow the prplist to cross a page boundary > > -When building the dsm-deallocate command, cdw10->NR was > used to count the number of ranges - starting with the value 1. Since NR is > a zero-based value, the count was decremented when command building was > completed. Unfortunately, when 256 ranges are specified, NR has a value of > 0 before decrementing. The value of 0 prevented the code from decrementing > NR. A change was made to use a local uint16 variable to cound the number of > ranges. > > SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based > value. A change was made to correctly set this as a zero-based value. > > SntiTranslateLogSense - A pointer to the data buffer is saved in the > srb extension. A change was made to also save the data buffer length in the > srb extension. The value is used when the data buffer is freed. > > > > nvmeStd.h > > -Increase size of dsmBuffer to allow for 16-byte alignment. > > -add dataBufferSize to store size of data buffer allocated for log > sense command. > > > > > > > > Tom Freeman > > > > Software Engineer, Device Manager and Driver Development > > HGST, a Western Digital company > > Thomas.Freeman at hgst.com > > 507-322-2311 > > > > > _______________________________________________ > nvmewin mailing list > nvmewin at lists.openfabrics.org > > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_nvmewin&d=AAICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=QOwFo5M7MYyQeT06CcSuSQHSUdSO20xC9GZe6-T9Svk&m=mWfi78Zo5vdb2DG993Dba37rsLday7RBzTm9-IJetnU&s=y2it_xUe2qb_N0eb0IHaUInfrvARtmSJ8w33jQetPVQ&e= > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: not available URL: From Alex.Chang at pmcs.com Fri Oct 10 17:05:26 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Sat, 11 Oct 2014 00:05:26 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Thank you very much, Parag. Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Friday, October 10, 2014 4:27 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, The changes look good. Thanks Parag Sheth On Thu, Oct 9, 2014 at 6:00 PM, Alex Chang > wrote: [cid:image001.gif at 01CFE4AC.448330F0] Hi Carolyn and Parag, Please let me know when you approve the patch. Thanks a lot, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Tuesday, October 07, 2014 10:17 AM To: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Here is the revised code code for the WMI/WPP + various changes. The password is “hgst1234” Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_nvmewin&d=AAICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=QOwFo5M7MYyQeT06CcSuSQHSUdSO20xC9GZe6-T9Svk&m=mWfi78Zo5vdb2DG993Dba37rsLday7RBzTm9-IJetnU&s=y2it_xUe2qb_N0eb0IHaUInfrvARtmSJ8w33jQetPVQ&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Sun Oct 12 21:32:21 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Mon, 13 Oct 2014 04:32:21 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Hi Carolyn, Please advise if you approve HGST’s patch. Thanks, Alex From: Parag Sheth [mailto:parag.sheth at seagate.com] Sent: Friday, October 10, 2014 4:27 PM To: Alex Chang Cc: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, The changes look good. Thanks Parag Sheth On Thu, Oct 9, 2014 at 6:00 PM, Alex Chang > wrote: [cid:image001.gif at 01CFE663.E09ADCF0] Hi Carolyn and Parag, Please let me know when you approve the patch. Thanks a lot, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Tuesday, October 07, 2014 10:17 AM To: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Here is the revised code code for the WMI/WPP + various changes. The password is “hgst1234” Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_nvmewin&d=AAICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=QOwFo5M7MYyQeT06CcSuSQHSUdSO20xC9GZe6-T9Svk&m=mWfi78Zo5vdb2DG993Dba37rsLday7RBzTm9-IJetnU&s=y2it_xUe2qb_N0eb0IHaUInfrvARtmSJ8w33jQetPVQ&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From carolyn.d.foster at intel.com Mon Oct 13 08:13:06 2014 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Mon, 13 Oct 2014 15:13:06 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Hi Alex, I approve this patch as well. Thanks, Carolyn From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Parag Sheth Sent: Friday, October 10, 2014 4:27 PM To: Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, The changes look good. Thanks Parag Sheth On Thu, Oct 9, 2014 at 6:00 PM, Alex Chang > wrote: [cid:image001.gif at 01CFE6BD.8219CC90] Hi Carolyn and Parag, Please let me know when you approve the patch. Thanks a lot, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Tuesday, October 07, 2014 10:17 AM To: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Here is the revised code code for the WMI/WPP + various changes. The password is “hgst1234” Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_nvmewin&d=AAICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=QOwFo5M7MYyQeT06CcSuSQHSUdSO20xC9GZe6-T9Svk&m=mWfi78Zo5vdb2DG993Dba37rsLday7RBzTm9-IJetnU&s=y2it_xUe2qb_N0eb0IHaUInfrvARtmSJ8w33jQetPVQ&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Mon Oct 13 09:00:12 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Mon, 13 Oct 2014 16:00:12 +0000 Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes In-Reply-To: References: <932a60f831004058b38554c6fdfe1348@BLUPR04MB673.namprd04.prod.outlook.com> <5495_1412702290_54342052_5495_8859_1_d5f310f6581b4554b8423f03b6bfb615@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Thank you very much, Carolyn. Alex From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Monday, October 13, 2014 8:13 AM To: Parag Sheth; Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, I approve this patch as well. Thanks, Carolyn From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Parag Sheth Sent: Friday, October 10, 2014 4:27 PM To: Alex Chang Cc: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Hi Alex, The changes look good. Thanks Parag Sheth On Thu, Oct 9, 2014 at 6:00 PM, Alex Chang > wrote: [cid:image001.gif at 01CFE6C3.F7B63B40] Hi Carolyn and Parag, Please let me know when you approve the patch. Thanks a lot, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Tuesday, October 07, 2014 10:17 AM To: Thomas Freeman; nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Here is the revised code code for the WMI/WPP + various changes. The password is “hgst1234” Changes to the original include: Use CACHING_MODE_PAGE instead of PMODE_CACHING_PAGE_EX Replacing RtlZeroMemory with memset Remove SetWmiDataBlock Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, September 24, 2014 1:55 PM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] HGST Changes to Enable WMI, Driver Trace and various fixes The attached zip file contains source code, project files and instructional documents for WMI and Driver tracing. The password is “hgst1234” We tested the changes on w7, w8 and w8.1 – 64-bit Testing included sdstress, iometer, quick and slow format: MBR and GPT and scsi compliance. Here is a description of the various changes: WMI (WindowsWmiDesc.pdf in the attached zip file gives details on using the change) nvme.vcxproj - Menu for "Mof Compiler" -Create Binary Mof File = "../../source/%(FileName).bmf" -WMI Syntax Check = Yes (-WMI) menu for Linker->Input -added $(DDK_LIB_PATH)\scsiwmi.lib directly in the file. This directive causes VS to create nvmeMofData.h: "$(IntDir)nvmeMofData.h" -u -h"$(IntDir)nvmeMofData.h" nvme.rc - Added "mofResource" statement nvmeMofData.bmf - Visual studio generates this file on every compile. Exists in the "source" directory. nvmeMofData.mof – Used to define WMI obects. Exists in the "source" directory nvmeMofData.h - Visual studio generates during compile. It is created based on nvmeMofData.mof. It exists in the project's intermediate directory (e.g. nvme/x64/Win7Debug). nvmeStd.c -NVMeFindAdapter - indicate that the driver is a wmiProvider and initialize the wmiContext. -NVMeBuildIo - Allow calls to WMI -NVMeStartIo - Allow calls to WMI nvmeStd.h -WmiLibContext - Added this structure to the device extension. The WmiLibContext structure provides registration information for a miniport driver's data and event blocks and defines entry points for the miniport driver's HwScsiWmiXxx callback routines. -WmiReqContext - Added this structure to the srb extension. When processing a WMI command, this structure is used to save context information for that particular command. precomp.h -added guiddef.h, scsiwmi.h, initguid.h, nvmeMofData.h, nmeWmi.h WPP - Driver tracing (WindowsTraceCollection.pdf in the attached zip file gives details on using the change) Relied heavily on the document "Using Storage Tracing Support In Miniport Drivers" - http://download.microsoft.com/download/8/e/8/8e85288f-b718-47d9-a7b2-af19d4c8f031/Storage-tracing-for-miniports.doc nvme.vcxproj - in the Menu for "Wpp Tracing" -Func to Gen Trace = StorStorPortDebugPrint(LEVEL,MSG,...) -Run Wpp Tracing = Yes -Scan Configuration Data = ..\..\source\nvme_tracing.h -Specify Template File = {km-StorDefault.tpl}*.tmh -Trace Kernel-Mode Components = Yes (-km) All c-files: Added include for "nvmeinit.tmh". A Visual Studio generated file to support driver tracing. nvmeStd.c - create a global variable for storing the trace context information. At cleanup time, the Port Driver calls a cleanup routine that uses this global variable. -DriverEntry - Include code to initialize driver tracing through wpp. -WppCleanupRoutine - Added code to be called by the port driver to clean up driver tracing. nvmeStd.h - -add function prototype for Wpp Cleanup routine. -Changed values for debug print levels to match values in evntrace.h INFO = TRACE_LEVEL_INFORMATION, WARNING = TRACE_LEVEL_WARNING, ERROR = TRACE_LEVEL_ERROR, TRACE = TRACE_LEVEL_VERBOSE precomp.h -added nvme_tracing.h Various Fixes nvme.vcxproj -Changed c/c++ compiler warning level to W3/WX nvmeSnti.c SntiTranslateReadCapacity16 - The caller’s response buffer was being cleared and set up without regard to its size. I put in changes to allocate a read capacity buffer on the stack. Once that temporary buffer was filled in, I used the response buffer size to determine how much data to copy into the caller’s response buffer. NOTE: It appears this same issue occurs in the handling of other scsi commands (e.g. in SntiTranslateDeviceIdentificationPage, if the response buffer was too short, setting the fields in pIdDescriptor would overrun the response buffer). This may be a fix to be implemented in the future. The issue in SntiTranslateReadCapacity16 issue was seen during testing. The potential issue with SntiTranslateDeviceIdentificationPage has not occurred in our testing. SnitTranslateUnmap -Changes to ensure the DataSetManagement buffer was always 16-byte aligned. -Use prp2 to allow the prplist to cross a page boundary -When building the dsm-deallocate command, cdw10->NR was used to count the number of ranges - starting with the value 1. Since NR is a zero-based value, the count was decremented when command building was completed. Unfortunately, when 256 ranges are specified, NR has a value of 0 before decrementing. The value of 0 prevented the code from decrementing NR. A change was made to use a local uint16 variable to cound the number of ranges. SntiTranslateWriteBuffer - The existing code set dword10 as a 1-based value. A change was made to correctly set this as a zero-based value. SntiTranslateLogSense - A pointer to the data buffer is saved in the srb extension. A change was made to also save the data buffer length in the srb extension. The value is used when the data buffer is freed. nvmeStd.h -Increase size of dsmBuffer to allow for 16-byte alignment. -add dataBufferSize to store size of data buffer allocated for log sense command. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openfabrics.org_mailman_listinfo_nvmewin&d=AAICAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=QOwFo5M7MYyQeT06CcSuSQHSUdSO20xC9GZe6-T9Svk&m=mWfi78Zo5vdb2DG993Dba37rsLday7RBzTm9-IJetnU&s=y2it_xUe2qb_N0eb0IHaUInfrvARtmSJ8w33jQetPVQ&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1117 bytes Desc: image001.gif URL: From Alex.Chang at pmcs.com Mon Oct 13 09:14:14 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Mon, 13 Oct 2014 16:14:14 +0000 Subject: [nvmewin] NVMe Windows DB Is LOCKED - Pushing HGST Patch For WMI And Driver Tracing Message-ID: Locking NVMe Windows DB. Thanks, Alex nvmewin mailing list nvmewin at lists.openfabrics.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Mon Oct 13 09:51:52 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Mon, 13 Oct 2014 16:51:52 +0000 Subject: [nvmewin] NVMe Windows DB Is UNLOCKED - Pushing Patch From HGST For WMI And Driver Tracing Message-ID: Dear all, Thank you for reviewing/testing the patch from HGST. Thank Tom and Moji from HGST for adding the patch. The patch had been pushed into the source base and a new tag called "Patch#29_WMI_Driver_Tracing" had been created under "tags" directory. In order to compile the newly added features, the VS 2013 project files had been modified and added under "VS2013_sln_proj" directory. Now, we start to support VS 2013 for our driver building. If you are still using older Visual Studio, you need to modify your project/solution files to include the WMI/driver tracing features. To bring up VS 2013 build environment, simply double-clicking the solution file under "VS2013_sln_proj" directory. Two documents (WindowsTraceCollection.pdf and WindowsWmiDesc.pdf) related to WMI and driver tracing are also added in "docs" directory for your reference. Samsung is scheduled to submit next patch after re-basing with most current sources in the repository. Hi Suman and Judy, You may send out your patch when it's available for the community to review and test. Thanks, Alex nvmewin mailing list nvmewin at lists.openfabrics.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From cheng.peng at memblaze.com Tue Oct 14 23:21:51 2014 From: cheng.peng at memblaze.com (cheng.peng at memblaze.com) Date: Wed, 15 Oct 2014 14:21:51 +0800 Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Message-ID: <201410151421511644424@memblaze.com> Hi all: should be wmilib.lib and scsiwmi.lib included in trunk\source\sources file? cheng.peng at memblaze.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From suman.p at samsung.com Wed Oct 15 05:59:32 2014 From: suman.p at samsung.com (SUMAN PRAKASH B) Date: Wed, 15 Oct 2014 12:59:32 +0000 (GMT) Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements Message-ID: <79.28.16067.4BF6E345@epcpsbgx4.samsung.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 201410151829952_XOK0LK7C.gif Type: image/gif Size: 13168 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: SamsungPatch2_v1_10152014.zip Type: application/octet-stream Size: 186196 bytes Desc: not available URL: From thomas.freeman at hgst.com Wed Oct 15 07:08:16 2014 From: thomas.freeman at hgst.com (Thomas Freeman) Date: Wed, 15 Oct 2014 14:08:16 +0000 Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib In-Reply-To: <201410151421511644424@memblaze.com> References: <201410151421511644424@memblaze.com> Message-ID: <5643b92d32fa42afb305453e406356e8@BLUPR04MB673.namprd04.prod.outlook.com> I did not update the sources file for those changes. My understanding is that with WDK 8.1 "sources" is not used. The project builds fine without these changes. Alex, do those changes need to be put into sources? If it is updated, nvmeWmi.c also needs to be added. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of cheng.peng at memblaze.com Sent: Wednesday, October 15, 2014 1:22 AM To: nvmewin Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Hi all: should be wmilib.lib and scsiwmi.lib included in trunk\source\sources file? ________________________________ cheng.peng at memblaze.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Wed Oct 15 09:23:19 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 15 Oct 2014 16:23:19 +0000 Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib In-Reply-To: <5643b92d32fa42afb305453e406356e8@BLUPR04MB673.namprd04.prod.outlook.com> References: <201410151421511644424@memblaze.com> <5643b92d32fa42afb305453e406356e8@BLUPR04MB673.namprd04.prod.outlook.com> Message-ID: Hi Peng, We had decided to migrate to VS 2013 and driver compiling and linking is done within VS 2013. Are you still using command prompt to build the driver after VS 2013 installed? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, October 15, 2014 7:08 AM To: cheng.peng at memblaze.com; nvmewin Subject: Re: [nvmewin] sources miss wmilib.lib and scsiwmi.lib I did not update the sources file for those changes. My understanding is that with WDK 8.1 "sources" is not used. The project builds fine without these changes. Alex, do those changes need to be put into sources? If it is updated, nvmeWmi.c also needs to be added. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of cheng.peng at memblaze.com Sent: Wednesday, October 15, 2014 1:22 AM To: nvmewin Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Hi all: should be wmilib.lib and scsiwmi.lib included in trunk\source\sources file? ________________________________ cheng.peng at memblaze.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Wed Oct 15 09:25:32 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 15 Oct 2014 16:25:32 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: Thank you very much, Judy and Suman, for the swift response and have the patch available. Dear all, Please start reviewing/testing the patch at your earliest convenience and provide your feedback. Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman [cid:image001.gif at 01CFE859.D481BF40] [Image removed by sender.] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ~WRD000.jpg Type: image/jpeg Size: 823 bytes Desc: ~WRD000.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 13168 bytes Desc: image001.gif URL: From cheng.peng at memblaze.com Wed Oct 15 18:48:14 2014 From: cheng.peng at memblaze.com (cheng.peng at memblaze.com) Date: Thu, 16 Oct 2014 09:48:14 +0800 Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib References: <201410151421511644424@memblaze.com>, <5643b92d32fa42afb305453e406356e8@BLUPR04MB673.namprd04.prod.outlook.com>, Message-ID: <201410160948137205114@memblaze.com> thank you for reply I used Nmake2MsBuild command to convert sources to vs2013 project before Now, I use directly msbuild command for nvme.sln cheng.peng at memblaze.com From: Alex Chang Date: 2014-10-16 00:23 To: Thomas Freeman; cheng.peng at memblaze.com; nvmewin Subject: RE: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Hi Peng, We had decided to migrate to VS 2013 and driver compiling and linking is done within VS 2013. Are you still using command prompt to build the driver after VS 2013 installed? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, October 15, 2014 7:08 AM To: cheng.peng at memblaze.com; nvmewin Subject: Re: [nvmewin] sources miss wmilib.lib and scsiwmi.lib I did not update the sources file for those changes. My understanding is that with WDK 8.1 “sources” is not used. The project builds fine without these changes. Alex, do those changes need to be put into sources? If it is updated, nvmeWmi.c also needs to be added. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of cheng.peng at memblaze.com Sent: Wednesday, October 15, 2014 1:22 AM To: nvmewin Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Hi all: should be wmilib.lib and scsiwmi.lib included in trunk\source\sources file? cheng.peng at memblaze.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Wed Oct 15 19:04:32 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Thu, 16 Oct 2014 02:04:32 +0000 Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib In-Reply-To: <201410160948137205114@memblaze.com> References: <201410151421511644424@memblaze.com>, <5643b92d32fa42afb305453e406356e8@BLUPR04MB673.namprd04.prod.outlook.com>, <201410160948137205114@memblaze.com> Message-ID: Thank you, Peng, for letting us know. Alex From: cheng.peng at memblaze.com [mailto:cheng.peng at memblaze.com] Sent: Wednesday, October 15, 2014 6:48 PM To: Alex Chang; Thomas Freeman; nvmewin Subject: Re: RE: [nvmewin] sources miss wmilib.lib and scsiwmi.lib thank you for reply I used Nmake2MsBuild command to convert sources to vs2013 project before Now, I use directly msbuild command for nvme.sln ________________________________ cheng.peng at memblaze.com From: Alex Chang Date: 2014-10-16 00:23 To: Thomas Freeman; cheng.peng at memblaze.com; nvmewin Subject: RE: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Hi Peng, We had decided to migrate to VS 2013 and driver compiling and linking is done within VS 2013. Are you still using command prompt to build the driver after VS 2013 installed? Thanks, Alex From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Thomas Freeman Sent: Wednesday, October 15, 2014 7:08 AM To: cheng.peng at memblaze.com; nvmewin Subject: Re: [nvmewin] sources miss wmilib.lib and scsiwmi.lib I did not update the sources file for those changes. My understanding is that with WDK 8.1 “sources” is not used. The project builds fine without these changes. Alex, do those changes need to be put into sources? If it is updated, nvmeWmi.c also needs to be added. Tom Freeman Software Engineer, Device Manager and Driver Development HGST, a Western Digital company Thomas.Freeman at hgst.com 507-322-2311 From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of cheng.peng at memblaze.com Sent: Wednesday, October 15, 2014 1:22 AM To: nvmewin Subject: [nvmewin] sources miss wmilib.lib and scsiwmi.lib Hi all: should be wmilib.lib and scsiwmi.lib included in trunk\source\sources file? ________________________________ cheng.peng at memblaze.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mithun.davis at keysight.com Fri Oct 17 03:12:39 2014 From: mithun.davis at keysight.com (mithun.davis at keysight.com) Date: Fri, 17 Oct 2014 10:12:39 +0000 Subject: [nvmewin] NVMe Windows driver copy progress issue Message-ID: Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun -------------- next part -------------- An HTML attachment was scrubbed... URL: From Uma.Parepalli at skhms.com Fri Oct 17 07:22:32 2014 From: Uma.Parepalli at skhms.com (Uma Parepalli) Date: Fri, 17 Oct 2014 14:22:32 +0000 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: References: Message-ID: <4e36adfc11c44ca68dda4ba1b4ed869d@N111XMB0240.skhms.com> Is this on Windows 7, 8 or 8.1 and debug or release driver? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of mithun.davis at keysight.com Sent: Friday, October 17, 2014 3:13 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows driver copy progress issue Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun The information contained in this e-mail is considered confidential of SK hynix memory solutions Inc. and intended only for the persons addressed or copied in this e-mail. Any unauthorized use, dissemination of the information, or copying of this message is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeb at bustrace.com Sat Oct 18 08:08:51 2014 From: mikeb at bustrace.com (Mike Berhan) Date: Sat, 18 Oct 2014 09:08:51 -0600 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: References: Message-ID: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> That's because you are dealing with buffered writes. Try using xcopy /j to perform your copy. http://technet.microsoft.com/en-us/library/cc771254(WS.10).aspx I wrote a simple batch file to copy a 512MB file from a RAM drive to an NVMe drive and do it 6 times. These are the timings: 1080 msec (with /j) 280 msec 310 msec 280 msec 2160 msec (with /j) 280 msec The 5th timing is double the 1st timing because it is now competing for bandwidth from the buffered writes that preceded it. Mike From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of mithun.davis at keysight.com Sent: Friday, October 17, 2014 4:13 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows driver copy progress issue Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun From mithun.davis at keysight.com Mon Oct 20 00:18:03 2014 From: mithun.davis at keysight.com (mithun.davis at keysight.com) Date: Mon, 20 Oct 2014 07:18:03 +0000 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: <4e36adfc11c44ca68dda4ba1b4ed869d@N111XMB0240.skhms.com> References: <4e36adfc11c44ca68dda4ba1b4ed869d@N111XMB0240.skhms.com> Message-ID: This is Win7. Release and Debug both have this problem. Regards, Mithun From: Uma Parepalli [mailto:Uma.Parepalli at skhms.com] Sent: Friday, 17 October, 2014 7:53 PM To: DAVIS,MITHUN (K-India,ex1); nvmewin at lists.openfabrics.org Subject: RE: NVMe Windows driver copy progress issue Is this on Windows 7, 8 or 8.1 and debug or release driver? Uma From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of mithun.davis at keysight.com Sent: Friday, October 17, 2014 3:13 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows driver copy progress issue Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun The information contained in this e-mail is considered confidential of SK hynix memory solutions Inc. and intended only for the persons addressed or copied in this e-mail. Any unauthorized use, dissemination of the information, or copying of this message is strictly prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete the original and any copies of this email. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mithun.davis at keysight.com Mon Oct 20 11:41:39 2014 From: mithun.davis at keysight.com (mithun.davis at keysight.com) Date: Mon, 20 Oct 2014 18:41:39 +0000 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> References: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> Message-ID: Hi Mike, I did try this out. When copying with /j the command returns only after copy is actually over while when this switch is not used, xcopy returns immediately. So I think you are right, the cause of the issue is buffering. Is there some way we can prevent buffering for specific drives or can we switch this off in the driver? Thanks for your help. Regards, Mithun -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Mike Berhan Sent: Saturday, 18 October, 2014 8:39 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] NVMe Windows driver copy progress issue That's because you are dealing with buffered writes. Try using xcopy /j to perform your copy. http://technet.microsoft.com/en-us/library/cc771254(WS.10).aspx I wrote a simple batch file to copy a 512MB file from a RAM drive to an NVMe drive and do it 6 times. These are the timings: 1080 msec (with /j) 280 msec 310 msec 280 msec 2160 msec (with /j) 280 msec The 5th timing is double the 1st timing because it is now competing for bandwidth from the buffered writes that preceded it. Mike From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of mithun.davis at keysight.com Sent: Friday, October 17, 2014 4:13 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows driver copy progress issue Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From mikeb at bustrace.com Mon Oct 20 14:45:55 2014 From: mikeb at bustrace.com (Mike Berhan) Date: Mon, 20 Oct 2014 15:45:55 -0600 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: References: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> Message-ID: <000501cfecaf$3a880b60$af982220$@bustrace.com> Regarding changes in the miniport, you could make the device look like it has removable media to the OS (Removable bit in STOR_DEVICE_CAPABILITIES and Inquiry page). If you do, then the "Policies" page for the disk device in device manager will allow you to enable "Quick removal" which states: "Disables write caching on the device and in Windows..." This will have other implications which may not be desirable, such as the device appearing in the taskbar eject media options. Experiment with it if you like. There may be other options that would do the same thing. What are you trying to accomplish? If it's benchmarking, why not use a benchmark app that accesses the physical disk directly? Mike -----Original Message----- From: mithun.davis at keysight.com [mailto:mithun.davis at keysight.com] Sent: Monday, October 20, 2014 12:42 PM To: mikeb at bustrace.com; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] NVMe Windows driver copy progress issue Hi Mike, I did try this out. When copying with /j the command returns only after copy is actually over while when this switch is not used, xcopy returns immediately. So I think you are right, the cause of the issue is buffering. Is there some way we can prevent buffering for specific drives or can we switch this off in the driver? Thanks for your help. Regards, Mithun -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Mike Berhan Sent: Saturday, 18 October, 2014 8:39 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] NVMe Windows driver copy progress issue That's because you are dealing with buffered writes. Try using xcopy /j to perform your copy. http://technet.microsoft.com/en-us/library/cc771254(WS.10).aspx I wrote a simple batch file to copy a 512MB file from a RAM drive to an NVMe drive and do it 6 times. These are the timings: 1080 msec (with /j) 280 msec 310 msec 280 msec 2160 msec (with /j) 280 msec The 5th timing is double the 1st timing because it is now competing for bandwidth from the buffered writes that preceded it. Mike From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of mithun.davis at keysight.com Sent: Friday, October 17, 2014 4:13 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows driver copy progress issue Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From mithun.davis at keysight.com Tue Oct 21 00:47:46 2014 From: mithun.davis at keysight.com (mithun.davis at keysight.com) Date: Tue, 21 Oct 2014 07:47:46 +0000 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: <000501cfecaf$3a880b60$af982220$@bustrace.com> References: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> <000501cfecaf$3a880b60$af982220$@bustrace.com> Message-ID: Hi Mike, I did try the following steps that you have mentioned. And it works fine in this case. One question though, any way to have the 'Quick Removal' set by default? Regards, Mithun -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Mike Berhan Sent: Tuesday, 21 October, 2014 3:16 AM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] NVMe Windows driver copy progress issue Regarding changes in the miniport, you could make the device look like it has removable media to the OS (Removable bit in STOR_DEVICE_CAPABILITIES and Inquiry page). If you do, then the "Policies" page for the disk device in device manager will allow you to enable "Quick removal" which states: "Disables write caching on the device and in Windows..." This will have other implications which may not be desirable, such as the device appearing in the taskbar eject media options. Experiment with it if you like. There may be other options that would do the same thing. What are you trying to accomplish? If it's benchmarking, why not use a benchmark app that accesses the physical disk directly? Mike -----Original Message----- From: mithun.davis at keysight.com [mailto:mithun.davis at keysight.com] Sent: Monday, October 20, 2014 12:42 PM To: mikeb at bustrace.com; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] NVMe Windows driver copy progress issue Hi Mike, I did try this out. When copying with /j the command returns only after copy is actually over while when this switch is not used, xcopy returns immediately. So I think you are right, the cause of the issue is buffering. Is there some way we can prevent buffering for specific drives or can we switch this off in the driver? Thanks for your help. Regards, Mithun -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Mike Berhan Sent: Saturday, 18 October, 2014 8:39 PM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] NVMe Windows driver copy progress issue That's because you are dealing with buffered writes. Try using xcopy /j to perform your copy. http://technet.microsoft.com/en-us/library/cc771254(WS.10).aspx I wrote a simple batch file to copy a 512MB file from a RAM drive to an NVMe drive and do it 6 times. These are the timings: 1080 msec (with /j) 280 msec 310 msec 280 msec 2160 msec (with /j) 280 msec The 5th timing is double the 1st timing because it is now competing for bandwidth from the buffered writes that preceded it. Mike From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of mithun.davis at keysight.com Sent: Friday, October 17, 2014 4:13 AM To: nvmewin at lists.openfabrics.org Subject: [nvmewin] NVMe Windows driver copy progress issue Hi, I am faced with a particular problem with the NVMe driver for Windows. I am using the 1.3 revision of the driver. When I copy a large file onto the device, it appears to have been copied in windows explorer but in the background the driver still seems to be copying it. The device being used is not very fast to respond which is the cause of the slow speed. Can anyone please help with why the file appears to be copied in windows explorer? I would like it to show the copy status as close to actual as possible. Thanks in advance for your help. Regards, Mithun _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From Alex.Chang at pmcs.com Tue Oct 21 13:20:29 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Tue, 21 Oct 2014 20:20:29 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   From mikeb at bustrace.com Wed Oct 22 14:16:50 2014 From: mikeb at bustrace.com (Mike Berhan) Date: Wed, 22 Oct 2014 15:16:50 -0600 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: References: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> <000501cfecaf$3a880b60$af982220$@bustrace.com> Message-ID: <001401cfee3d$7faca3b0$7f05eb10$@bustrace.com> The answer to your question can be found here: http://blogs.msdn.com/b/xiz/archive/2013/11/04/removal-policy-for-a-device.a spx Mike -----Original Message----- From: mithun.davis at keysight.com [mailto:mithun.davis at keysight.com] Sent: Tuesday, October 21, 2014 1:48 AM To: mikeb at bustrace.com; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] NVMe Windows driver copy progress issue Hi Mike, I did try the following steps that you have mentioned. And it works fine in this case. One question though, any way to have the 'Quick Removal' set by default? Regards, Mithun From carolyn.d.foster at intel.com Wed Oct 22 16:28:56 2014 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Wed, 22 Oct 2014 23:28:56 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. As the patch is currently written I am not comfortable approving it. This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From Alex.Chang at pmcs.com Thu Oct 23 11:05:46 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Thu, 23 Oct 2014 18:05:46 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: Hi Suman, It seems to me that both polledMode and hwResetInProg are used for exactly same purpose. Can you also address the reason why you added polledMode? Thank you! Alex -----Original Message----- From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. As the patch is currently written I am not comfortable approving it. This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From judy.brock at ssi.samsung.com Thu Oct 23 11:06:15 2014 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Thu, 23 Oct 2014 18:06:15 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A516B660D3A@SSIEXCH-MB3.ssi.samsung.com> Hi Alex, We're fine with both the changes you suggest below (Suman's on vacation this week so I'm answering on his behalf - we discussed it). Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From judy.brock at ssi.samsung.com Thu Oct 23 11:07:51 2014 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Thu, 23 Oct 2014 18:07:51 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A516B660D61@SSIEXCH-MB3.ssi.samsung.com> Hi, Suman is on vacation this week. I will respond to your email below, Alex, and Carolyn's as well today. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Thursday, October 23, 2014 11:06 AM To: Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, It seems to me that both polledMode and hwResetInProg are used for exactly same purpose. Can you also address the reason why you added polledMode? Thank you! Alex -----Original Message----- From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. As the patch is currently written I am not comfortable approving it. This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From Alex.Chang at pmcs.com Thu Oct 23 11:08:59 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Thu, 23 Oct 2014 18:08:59 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A516B660D61@SSIEXCH-MB3.ssi.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B660D61@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Thank you very much, Judy. Alex -----Original Message----- From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 11:08 AM To: Alex Chang; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Hi, Suman is on vacation this week. I will respond to your email below, Alex, and Carolyn's as well today. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Thursday, October 23, 2014 11:06 AM To: Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, It seems to me that both polledMode and hwResetInProg are used for exactly same purpose. Can you also address the reason why you added polledMode? Thank you! Alex -----Original Message----- From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. As the patch is currently written I am not comfortable approving it. This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From judy.brock at ssi.samsung.com Thu Oct 23 18:45:10 2014 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Fri, 24 Oct 2014 01:45:10 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From judy.brock at ssi.samsung.com Fri Oct 24 02:18:35 2014 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Fri, 24 Oct 2014 09:18:35 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A516B66150B@SSIEXCH-MB3.ssi.samsung.com> Alex, I need to wait for Suman to come back to make sure we're in agreement about this but it looks to me like we don't need both. We could possibly replace all occurrences of both with a single var called polledResetInProg (cuz not all reset ops/reinit ops are polled). And eliminate the duplicate definition in nvmestd.h and the duplicate initialization in nvmestd.c in NVMeReInitializeController(). Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Thursday, October 23, 2014 11:06 AM To: Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, It seems to me that both polledMode and hwResetInProg are used for exactly same purpose. Can you also address the reason why you added polledMode? Thank you! Alex -----Original Message----- From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. As the patch is currently written I am not comfortable approving it. This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From judy.brock at ssi.samsung.com Fri Oct 24 02:39:13 2014 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Fri, 24 Oct 2014 09:39:13 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A516B661540@SSIEXCH-MB3.ssi.samsung.com> Hi Carolyn et al, A couple more thoughts. There are other options; in fact we assumed there would be discussion on the reflector if folks were interested in revisiting the design we settled on (as you are). One option is to remove the driver support for (NVMeAdapterControl-> ScsiRestartAdapter) entirely and instead when resuming from hibernation for example, allow NVMeStartio() to process the POWER SRB and handle the Power Up operation there. If we do that, we are no longer tasked with doing all the work synchronously. We can launch the reinit state machine, return from NVMeStartio, and complete the SRB later asynchronously when the reinit state machine has finished. This is the more "modern" approach. The reason we hesitated to go down this path is because we felt it might be viewed as a high risk change to remove support for (NVMeAdapterControl-> ScsiRestartAdapter) entirely since it's been there for such a long time. In fact, it might be a fine low-risk thing to do - we just didn't know how the team would feel about it or how it might change the timing of the driver. But the team should consider it. Another thing I noticed tonight that might be do-able would be to collapse a couple of the paths in NVMeRunningStartAttempt()into a single path. The polledmode loop is not that different from the dump mode loop. The dump mode loop is actually doing the same thing behind the scenes - it is calling the interrupt handler directly for all Admin cmds (due to lack of timer services) and the isr is in turn calling IoCompletionDpcRoutine directly. So we have two polling loops that are very similar and we may not need both. Thanks, Judy From: Judy Brock-SSI Sent: Thursday, October 23, 2014 6:45 PM To: 'Foster, Carolyn D'; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Fri Oct 24 08:56:58 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Fri, 24 Oct 2014 15:56:58 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A516B66150B@SSIEXCH-MB3.ssi.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B66150B@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Sounds good to me. Thank you very much, Judy, for addressing the concerns. Alex -----Original Message----- From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Friday, October 24, 2014 2:19 AM To: Alex Chang; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Alex, I need to wait for Suman to come back to make sure we're in agreement about this but it looks to me like we don't need both. We could possibly replace all occurrences of both with a single var called polledResetInProg (cuz not all reset ops/reinit ops are polled). And eliminate the duplicate definition in nvmestd.h and the duplicate initialization in nvmestd.c in NVMeReInitializeController(). Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Thursday, October 23, 2014 11:06 AM To: Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, It seems to me that both polledMode and hwResetInProg are used for exactly same purpose. Can you also address the reason why you added polledMode? Thank you! Alex -----Original Message----- From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. As the patch is currently written I am not comfortable approving it. This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone,   We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123   Please find the change description below -   1. There are multiple paths in the driver that reset the controller and execute the initialization state machine.  Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed -  but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter.   Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE.   3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We  fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED.   4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed.     Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter   Thanks, Suman   _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From Alex.Chang at pmcs.com Mon Oct 27 20:56:56 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Tue, 28 Oct 2014 03:56:56 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Hi Carolyn, Please let us know what you think. Thanks, Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 6:45 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From mithun.davis at keysight.com Tue Oct 28 02:03:30 2014 From: mithun.davis at keysight.com (mithun.davis at keysight.com) Date: Tue, 28 Oct 2014 09:03:30 +0000 Subject: [nvmewin] NVMe Windows driver copy progress issue In-Reply-To: <001401cfee3d$7faca3b0$7f05eb10$@bustrace.com> References: <002601cfeae5$6d6ffd60$484ff820$@bustrace.com> <000501cfecaf$3a880b60$af982220$@bustrace.com> <001401cfee3d$7faca3b0$7f05eb10$@bustrace.com> Message-ID: Thanks Mike. Will try this out. Regards, Mithun -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Mike Berhan Sent: Thursday, 23 October, 2014 2:47 AM To: nvmewin at lists.openfabrics.org Subject: Re: [nvmewin] NVMe Windows driver copy progress issue The answer to your question can be found here: http://blogs.msdn.com/b/xiz/archive/2013/11/04/removal-policy-for-a-device.a spx Mike -----Original Message----- From: mithun.davis at keysight.com [mailto:mithun.davis at keysight.com] Sent: Tuesday, October 21, 2014 1:48 AM To: mikeb at bustrace.com; nvmewin at lists.openfabrics.org Subject: RE: [nvmewin] NVMe Windows driver copy progress issue Hi Mike, I did try the following steps that you have mentioned. And it works fine in this case. One question though, any way to have the 'Quick Removal' set by default? Regards, Mithun _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin From carolyn.d.foster at intel.com Tue Oct 28 16:24:23 2014 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Tue, 28 Oct 2014 23:24:23 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Hi Alex, Judy and Suman, I have completed my testing of the proposed patch and verified that it works. I also agree with Judy's comments below. Since we're now calling a function that is normally called in a DPC, for clarity, I would like to see a new function that calls the DPC routine instead. You could give the new function a name that indicates that it will handle command completions in polled mode. Then that new function would call the DPC routine directly, and the new function would be called from RunningStartAttempt. This way it doesn't look like the RunningStartAttempt routine is doing anything with DPCs. Thanks, Carolyn From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 27, 2014 8:57 PM To: Judy Brock-SSI; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Please let us know what you think. Thanks, Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 6:45 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From judy.brock at ssi.samsung.com Tue Oct 28 17:24:48 2014 From: judy.brock at ssi.samsung.com (Judy Brock-SSI) Date: Wed, 29 Oct 2014 00:24:48 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> Message-ID: <36E8D38D6B771A4BBDB1C0D800158A516B663F51@SSIEXCH-MB3.ssi.samsung.com> Hi Carolyn, Thank you for your feedback. It is a good idea to eliminate confusion over the currently-named routine - IoCompletionDpcRoutine - which does indeed imply running as a DPC. We could either create a new function, as you suggest or alternatively, we could just rename IoCompletionDpcRoutine directly to for example, "ProcessIoCompletions" or just "IoCompletionRoutine" and then indicate at the top of that routine that it can either be scheduled to run as a DPC or called directly, depending on context. If we take the word "Dpc" out of it, I think it might eliminate all potential confusion. If it is ok with you, I think renaming IoCompletionDpcRoutine to no longer imply the method by which its invoked might be preferable as this would also cover/clarify the case where, in dump mode, NVMeIsrMsix is currently calling IoCompletionDpcRoutine directly. If you still prefer a new routine, to cover both cases above, the name should imply the direct call nature of the function rather than polled mode per se since, if we call it from the ISR it could lead to the opposite confusion - a function that implies no ISR involvement being called from the ISR itself. We could: Create new function called ImmediateCompletionProcessing (or some other name that implies non-deferred processing) which in turn would call IoCompletionDpcRoutine. But again, I think renaming IoCompletionDpcRoutine is better because no matter what we name a new function, if it still calls a routine with the word "Dpc" in it, it will still have the potential to create confusion in (especially new) readers minds. Thanks, Judy From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Tuesday, October 28, 2014 4:24 PM To: Alex Chang; Judy Brock-SSI; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Alex, Judy and Suman, I have completed my testing of the proposed patch and verified that it works. I also agree with Judy's comments below. Since we're now calling a function that is normally called in a DPC, for clarity, I would like to see a new function that calls the DPC routine instead. You could give the new function a name that indicates that it will handle command completions in polled mode. Then that new function would call the DPC routine directly, and the new function would be called from RunningStartAttempt. This way it doesn't look like the RunningStartAttempt routine is doing anything with DPCs. Thanks, Carolyn From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 27, 2014 8:57 PM To: Judy Brock-SSI; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Please let us know what you think. Thanks, Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 6:45 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From suman.p at samsung.com Wed Oct 29 05:01:51 2014 From: suman.p at samsung.com (SUMAN PRAKASH B) Date: Wed, 29 Oct 2014 12:01:51 +0000 (GMT) Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements Message-ID: An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 201410291731423_XOK0LK7C.gif Type: image/gif Size: 13168 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: SamsungPatch2_v2_10292014.zip Type: application/octet-stream Size: 186197 bytes Desc: not available URL: From Alex.Chang at pmcs.com Wed Oct 29 08:34:28 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 29 Oct 2014 15:34:28 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A516B663F51@SSIEXCH-MB3.ssi.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B663F51@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Hi Suman, Please revise the patch accordingly and send it out for final review/test as soon as possible. In addition, if you more time on hot plug patch, I can send out my Format NVM patch after bus reset patch. Please advise. Thank you ! Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Tuesday, October 28, 2014 5:25 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Thank you for your feedback. It is a good idea to eliminate confusion over the currently-named routine - IoCompletionDpcRoutine - which does indeed imply running as a DPC. We could either create a new function, as you suggest or alternatively, we could just rename IoCompletionDpcRoutine directly to for example, "ProcessIoCompletions" or just "IoCompletionRoutine" and then indicate at the top of that routine that it can either be scheduled to run as a DPC or called directly, depending on context. If we take the word "Dpc" out of it, I think it might eliminate all potential confusion. If it is ok with you, I think renaming IoCompletionDpcRoutine to no longer imply the method by which its invoked might be preferable as this would also cover/clarify the case where, in dump mode, NVMeIsrMsix is currently calling IoCompletionDpcRoutine directly. If you still prefer a new routine, to cover both cases above, the name should imply the direct call nature of the function rather than polled mode per se since, if we call it from the ISR it could lead to the opposite confusion - a function that implies no ISR involvement being called from the ISR itself. We could: Create new function called ImmediateCompletionProcessing (or some other name that implies non-deferred processing) which in turn would call IoCompletionDpcRoutine. But again, I think renaming IoCompletionDpcRoutine is better because no matter what we name a new function, if it still calls a routine with the word "Dpc" in it, it will still have the potential to create confusion in (especially new) readers minds. Thanks, Judy From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Tuesday, October 28, 2014 4:24 PM To: Alex Chang; Judy Brock-SSI; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Alex, Judy and Suman, I have completed my testing of the proposed patch and verified that it works. I also agree with Judy's comments below. Since we're now calling a function that is normally called in a DPC, for clarity, I would like to see a new function that calls the DPC routine instead. You could give the new function a name that indicates that it will handle command completions in polled mode. Then that new function would call the DPC routine directly, and the new function would be called from RunningStartAttempt. This way it doesn't look like the RunningStartAttempt routine is doing anything with DPCs. Thanks, Carolyn From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 27, 2014 8:57 PM To: Judy Brock-SSI; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Please let us know what you think. Thanks, Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 6:45 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alex.Chang at pmcs.com Wed Oct 29 08:37:53 2014 From: Alex.Chang at pmcs.com (Alex Chang) Date: Wed, 29 Oct 2014 15:37:53 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <14308_1414584116_5450D734_14308_11885_1_62.08.14702.F27D0545@epcpsbgx3.samsung.com> References: <14308_1414584116_5450D734_14308_11885_1_62.08.14702.F27D0545@epcpsbgx3.samsung.com> Message-ID: Thank you very much, Suman. Dear all, Please review/test the revised patch at your earliest convenience and provide feedback if you have any. I plan to collect approval next Monday. Thanks, Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 29, 2014 5:02 AM To: Judy Brock-SSI; Foster, Carolyn D; Alex Chang; nvmewin at lists.openfabrics.org Subject: Re: RE: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Alex, I have revised the patch with the following review comments incorporated - 1. Both polledMode and hwResetInProg are used for exactly same purpose. [Suman] As suggested by Judy, replaced all occurrences of hwResetInProg and polledMode with a single variable polledResetInProg. 2. There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant. [Suman] Changed code to call the StorPortResume(pAE) in Line2434 of nvmestd.c only when pAE->DriverState.NextDriverState != NVMeStartComplete. 3. To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } [Suman] Corrected as per suggestion. 4. Rename IoCompletionDpcRoutine to avoid confusion, since this routine will be called in both DPC and polled mode. [Suman] As suggested by Judy, renamed this routine to IoCompletionRoutine and added comment in header section that this routine can either be scheduled to run as a DPC or called directly. Please find attached the revised patch. Password is samsung123. Thanks all for reviewing. Thanks, Suman ------- Original Message ------- Sender : Judy Brock-SSI Date : Oct 29, 2014 05:24 (GMT+05:00) Title : RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Thank you for your feedback. It is a good idea to eliminate confusion over the currently-named routine – IoCompletionDpcRoutine - which does indeed imply running as a DPC. We could either create a new function, as you suggest or alternatively, we could just rename IoCompletionDpcRoutine directly to for example, “ProcessIoCompletions” or just “IoCompletionRoutine” and then indicate at the top of that routine that it can either be scheduled to run as a DPC or called directly, depending on context. If we take the word “Dpc” out of it, I think it might eliminate all potential confusion. If it is ok with you, I think renaming IoCompletionDpcRoutine to no longer imply the method by which its invoked might be preferable as this would also cover/clarify the case where, in dump mode, NVMeIsrMsix is currently calling IoCompletionDpcRoutine directly. If you still prefer a new routine, to cover both cases above, the name should imply the direct call nature of the function rather than polled mode per se since, if we call it from the ISR it could lead to the opposite confusion – a function that implies no ISR involvement being called from the ISR itself. We could: Create new function called ImmediateCompletionProcessing (or some other name that implies non-deferred processing) which in turn would call IoCompletionDpcRoutine. But again, I think renaming IoCompletionDpcRoutine is better because no matter what we name a new function, if it still calls a routine with the word “Dpc” in it, it will still have the potential to create confusion in (especially new) readers minds. Thanks, Judy From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Tuesday, October 28, 2014 4:24 PM To: Alex Chang; Judy Brock-SSI; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Alex, Judy and Suman, I have completed my testing of the proposed patch and verified that it works. I also agree with Judy’s comments below. Since we’re now calling a function that is normally called in a DPC, for clarity, I would like to see a new function that calls the DPC routine instead. You could give the new function a name that indicates that it will handle command completions in polled mode. Then that new function would call the DPC routine directly, and the new function would be called from RunningStartAttempt. This way it doesn’t look like the RunningStartAttempt routine is doing anything with DPCs. Thanks, Carolyn From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 27, 2014 8:57 PM To: Judy Brock-SSI; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Please let us know what you think. Thanks, Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 6:45 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That’s because at runtime, we don’t want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses – i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don’t want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths – we want to poll. That’s why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can’t happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( … if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren’t enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn’t work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won’t. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin [cid:image001.gif at 01CFF353.9299C5A0] [Image removed by sender.] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ~WRD000.jpg Type: image/jpeg Size: 823 bytes Desc: ~WRD000.jpg URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 13168 bytes Desc: image001.gif URL: From carolyn.d.foster at intel.com Wed Oct 29 08:37:16 2014 From: carolyn.d.foster at intel.com (Foster, Carolyn D) Date: Wed, 29 Oct 2014 15:37:16 +0000 Subject: [nvmewin] Samsung Patch for Bus Reset Enhancements In-Reply-To: <36E8D38D6B771A4BBDB1C0D800158A516B663F51@SSIEXCH-MB3.ssi.samsung.com> References: <8810_1413377978_543E6FBA_8810_9358_1_A9.28.16067.4BF6E345@epcpsbgx4.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B661411@SSIEXCH-MB3.ssi.samsung.com> <36E8D38D6B771A4BBDB1C0D800158A516B663F51@SSIEXCH-MB3.ssi.samsung.com> Message-ID: Hi Judy, I'm fine with your suggestion to rename the completion routine. Thanks! Carolyn From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Tuesday, October 28, 2014 5:25 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Thank you for your feedback. It is a good idea to eliminate confusion over the currently-named routine - IoCompletionDpcRoutine - which does indeed imply running as a DPC. We could either create a new function, as you suggest or alternatively, we could just rename IoCompletionDpcRoutine directly to for example, "ProcessIoCompletions" or just "IoCompletionRoutine" and then indicate at the top of that routine that it can either be scheduled to run as a DPC or called directly, depending on context. If we take the word "Dpc" out of it, I think it might eliminate all potential confusion. If it is ok with you, I think renaming IoCompletionDpcRoutine to no longer imply the method by which its invoked might be preferable as this would also cover/clarify the case where, in dump mode, NVMeIsrMsix is currently calling IoCompletionDpcRoutine directly. If you still prefer a new routine, to cover both cases above, the name should imply the direct call nature of the function rather than polled mode per se since, if we call it from the ISR it could lead to the opposite confusion - a function that implies no ISR involvement being called from the ISR itself. We could: Create new function called ImmediateCompletionProcessing (or some other name that implies non-deferred processing) which in turn would call IoCompletionDpcRoutine. But again, I think renaming IoCompletionDpcRoutine is better because no matter what we name a new function, if it still calls a routine with the word "Dpc" in it, it will still have the potential to create confusion in (especially new) readers minds. Thanks, Judy From: Foster, Carolyn D [mailto:carolyn.d.foster at intel.com] Sent: Tuesday, October 28, 2014 4:24 PM To: Alex Chang; Judy Brock-SSI; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Alex, Judy and Suman, I have completed my testing of the proposed patch and verified that it works. I also agree with Judy's comments below. Since we're now calling a function that is normally called in a DPC, for clarity, I would like to see a new function that calls the DPC routine instead. You could give the new function a name that indicates that it will handle command completions in polled mode. Then that new function would call the DPC routine directly, and the new function would be called from RunningStartAttempt. This way it doesn't look like the RunningStartAttempt routine is doing anything with DPCs. Thanks, Carolyn From: Alex Chang [mailto:Alex.Chang at pmcs.com] Sent: Monday, October 27, 2014 8:57 PM To: Judy Brock-SSI; Foster, Carolyn D; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Please let us know what you think. Thanks, Alex From: Judy Brock-SSI [mailto:judy.brock at ssi.samsung.com] Sent: Thursday, October 23, 2014 6:45 PM To: Foster, Carolyn D; Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Subject: RE: Samsung Patch for Bus Reset Enhancements Hi Carolyn, Replies inline below in blue. Thanks, Judy -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Foster, Carolyn D Sent: Wednesday, October 22, 2014 4:29 PM To: Alex Chang; suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, I have some feedback in addition to Alex's comments. I believe there is an issue with the loop that was added to NVMeRunningStartAttempt. The issue is that IoCompletionDpcRoutine was never meant to be called directly. It was architected and designed to always run from a DPC. [Judy] That's because at runtime, we don't want to be doing time-consuming request-completion work in the ISR. Therefore the work is offloaded to a DPC which runs at a lower IRQL. However, the work we need to do to process cmd completions is fixed - there is actually no innate architectural design impediment in the routine itself to calling this routine directly in the two scenarios our patch addresses - i.e., those situations where by architectural definition we are expected to finish all work before returning to the caller (and in our case, that includes sending and completing multiple commands in our init state machine ).. Those scenarios are the two that Suman listed in the change notes: a) NVMeResetBus b) NVMeAdapterControl-> ScsiRestartAdapter By design, we don't want to schedule a DPC to handle completions for the commands generated by the init state machine in these 2 reset paths - we want to poll. That's why we make the direct call instead. It's possible that a command from the init state machine could generate an interrupt and run the IoCompletionDpcRoutine before it can be called in RunningStartAttempt. [Judy] This can't happen. If an interrupt is generated on behalf of a command from the init state machine during the first scenario above (NVMeResetBus), the hwResetInProg flag at the top of the ISR causes us to return immediately: NVMeIsrMsix ( ... if (pAE->hwResetInProg) return TRUE; The second scenario above (NVMeAdapterControl-> ScsiRestartAdapter) is not interrupt-driven by definition. That is, at the time it is called, interrupts aren't enabled. But even if it they were, the hwResetInProg flag would catch it. A better solution would be to have a loop similar to the one at the end of NVMePassiveInitialize where RunningStartAttempt is called, and is followed by a loop that waits for the state machine to complete. [Judy] This is actually the first approach we took and were intending to use but we found it didn't work. The reason was the loop you refer to is periodic timer-driven but the timer was not getting scheduled in the NVMeAdapterControl-> ScsiRestartAdapter path as there is no timer available at that point. The reason this is not an issue for the current OFA driver is because we launch the state machine but then return from the call to NVMeAdapterControl and let the state machine run asynchronously and complete outside of that context (violates the spec). As the patch is currently written I am not comfortable approving it.This change to wait for the state machine's completion could be made in the new ReinitializeController function, and then you wouldn't need the changes to RunningStartAttempt or any of the polledmode code. [Judy] The approach you propose will not work for the reason explained above. Again, we too had first hoped it would but it won't. Hence we went to a polled-mode model. Since we have to finish all work before returning anyway and since reset bus is not a performance path, there is no downside to polling. Thanks, Carolyn -----Original Message----- From: nvmewin-bounces at lists.openfabrics.org [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Alex Chang Sent: Tuesday, October 21, 2014 1:20 PM To: suman.p at samsung.com; nvmewin at lists.openfabrics.org Cc: cgps at samsung.com Subject: Re: [nvmewin] Samsung Patch for Bus Reset Enhancements Hi Suman, (1) There is a call of StorPortResume(pAE) in Line2434 of nvmestd.c, which is redundant because, when NextDriverState is NVMeStartComplete, in the end of NVMeRunning, StorPortResume had been called already. (2) To comply with our agreed coding style and make the logic easier, may I suggest changing Line#184 of nvmestat.c to: if (pAE->ntldrDump == FALSE) { if (pAE->polledMode == FALSE) { NVMeRunning(pAE); } else { /* * we poll if we're launching the reinit state machine from HwStorResetBus * or HwStorAdapterControl->ScsiRestartAdapter path */ NVMeRunning(pAE); /* TO val is based on CAP register plus a few, 5, seconds to init post RDY */ passiveTimeout = pAE->uSecCrtlTimeout + (STORPORT_TIMER_CB_us * MICRO_TO_SEC); ... return (pAE->DriverState.NextDriverState == NVMeStartComplete) ? TRUE : FALSE; } } else { PRES_MAPPING_TBL pRMT = &pAE->ResMapTbl; ..... } Thank you! Alex From: SUMAN PRAKASH B [mailto:suman.p at samsung.com] Sent: Wednesday, October 15, 2014 6:00 AM To: nvmewin at lists.openfabrics.org Cc: Alex Chang; cgps at samsung.com Subject: Samsung Patch for Bus Reset Enhancements Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Date: %%SENT_DATE%% Subject: Suspect Message Quarantined WARNING: The virus scanner was unable to scan an attachment in an email message sent to you. This attachment could possibly contain viruses or other malicious programs. The attachment could not be scanned for the following reasons: %%DESC%% The full message and the attachment have been stored in the quarantine. The identifier for this message is '%%QID%%'. Access the quarantine at: https://puremessage.pmc-sierra.bc.ca:28443/ For more information on PMC's Anti-Spam system: http://pmc-intranet/wiki/index.php/Outlook:Anti-Spam_FAQ IT Services PureMessage Admin Hi Everyone, We have a patch for the Bus Reset Enhancements. Please find attached the source code. The password is samsung123 Please find the change description below - 1. There are multiple paths in the driver that reset the controller and execute the initialization state machine. Our patch is not concerned with the majority of those paths. Aside from a few additional isolated modifications, our patch focuses on the two paths that are supposed to be synchronous -i.e. they should not return to caller until all work is completed - but which currently are not so. They are: a) NVMeResetBus (and) b) NVMeAdapterControl-> ScsiRestartAdapter We have introduced a new routine NVMeReInitializeController(), which will be invoked from NVMeReseBus() and NVMeAdapterControl() - ScsiRestartAdapter. This routine will reset and initialize the controller and then complete the requests. It will not return until the initialization state machine is complete. We disallow processing of any SRB in NVMeStartIo() when NextDriverState != NVMeStateComplete. In this way we direct the PowerUp operations to be executed in NVMeAdapterControl() - ScsiRestartAdapter only. When resuming from hibernation for example, NVMeStartio() will not process the POWER SRB. Instead, the Power Up operations will be invoked in NVMeAdapterControl()->ScsiRestartAdapter. Additionally , Miniport drivers should disregard requests to reset the bus when ntldrDump is set to TRUE in NvmeResetBus(). But current implementation processes this request. 2. When pAE->ntldrDump is TRUE, in the NVMeMapCore2Queue() routine, the pPGT value is NULL. Hence a BSOD occurs when executing ULONG coreNum = (ULONG)(pPN->Number + pPGT->BaseProcessor). We fixed the problem by moving access to pPGT when ntldrDump is FALSE. 3. In ProcessIo(), when IoStatus is set to NOT_SUBMITTED, the SRB is not completed. Due to this, a BSOD was occuring when executing WHCK test "DP WLK - Hot-Add - Device test". We fixed the problem by changing the code to complete SRB when IoStatus is NOT_SUBMITTED. 4. We changed the use of StorPortBusy()/StorPortReady() to StorPortPause()/StorPortResume(), since StorPortBusy() will not prevent new IOS from coming in once the current ones in the driver have been completed. Tested the following on Win7 and Windows 2012R2. - WHCK - Install/Uninstall, Enable/Disable, FS Format - Hibernation/Resume, Sleep/Resume - IOmeter Thanks, Suman _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin _______________________________________________ nvmewin mailing list nvmewin at lists.openfabrics.org http://lists.openfabrics.org/mailman/listinfo/nvmewin -------------- next part -------------- An HTML attachment was scrubbed... URL: