[nvmewin] Compatibility issue with 1.5 version nvme driver

Tue Jun 27 16:31:08 PDT 2017

Hi Justina,

Could you please reply to the latest thread emails where Tom and I are discussing the possible solutions.

After the discussion, I'm ok with acquiring the StartIo lock as long as it's in a new DPC spawned by the IOCompletionDpcRoutine.

Please also report the results of the unit tests previously outlined.

Thanks...
Ray

From: Justina Lai [mailto:justina_lai at phison.com]
Sent: Tuesday, June 27, 2017 3:11 AM
To: Robles, Raymond C <raymond.c.robles at intel.com>
Cc: nvmewin at lists.openfabrics.org; Larry Li <larry_li at phison.com>
Subject: RE: Compatibility issue with 1.5 version nvme driver

Dear Raymond,

Sorry for late reply.

Option-1
Please refer to attached nvmeSnti.c
Modification: Judge if execute process() SpinLock by SntiTranslateModeSenseResponse().

Option-2
Please refer to attached nvmeStd.c
Modification: Directly mark SpinLock/ ReleaseSpinLock in IoCompletionRoutine()

Thanks!

Best Regards,
Justina Lai
#5707

From: Robles, Raymond C [mailto:raymond.c.robles at intel.com]
Sent: Friday, June 23, 2017 7:20 AM
To: Justina Lai <justina_lai at phison.com<mailto:justina_lai at phison.com>>
Cc: nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>; Larry Li <larry_li at phison.com<mailto:larry_li at phison.com>>
Subject: RE: Compatibility issue with 1.5 version nvme driver

Hi Justina,

Intel has reviewed your and tested your patch. Please see the following comments:

  *   Need to remove code blocks in #if 0
  *   When multiple cpu cores are mapped with single queue, the patch is removing startIoLock from processIO and startIoLock used only in IOCompletionDpcRoutine:
  *   This will block NvmeStartIo when driver is processing a completion entry.
  *   A couple of options:
     *   Indicate in nvmeSnti.c when handling operations that require more than one command to the drive, and have that locking needs passed to ProcessIo appropriately. It does not seem plausible to isolate the changes to nvmeIO.c only.
     *   You could also instead remove startIoLock from IoCompletionDpcRoutine while keeping startIoLock at processIo. This way we can avoid double lock and also we can make NvmeStartIo and IoCompletionDpcRoutine parallel.

The key takeaway is that StartIo and IoCompletionDpcRoutine must be able to run concurrently with no deadlocks. As you've coded the patch, a deadlock is possible. Please revise your patch to account for this scenario.

Thanks...
Ray

From: nvmewin [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Robles, Raymond C
Sent: Thursday, June 22, 2017 3:55 PM
To: Justina Lai <justina_lai at phison.com<mailto:justina_lai at phison.com>>
Cc: nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: Re: [nvmewin] Compatibility issue with 1.5 version nvme driver

Hi Justina,

Please go the Open Fabrics Alliance website and subscribe to our reflector on this page:  http://lists.openfabrics.org/mailman/listinfo/nvmewin

The main NVMe WG page can be found here: https://www.openfabrics.org/index.php/working-groups.html

Thanks...
Ray

From: Justina Lai [mailto:justina_lai at phison.com]
Sent: Monday, June 19, 2017 11:45 PM
To: Robles, Raymond C <raymond.c.robles at intel.com<mailto:raymond.c.robles at intel.com>>
Cc: nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: RE: Compatibility issue with 1.5 version nvme driver

Dear Raymond,

Yes, I have received your mail regarding unit test.
We are now carrying on the test and hope to finish it within this week.

May I know how to officially subscribe to the reflector email list?
I saw my mail is already in the Non-digested Members of nvmewin. Should I set as digested mode?

Thank you!

Best Regards,
Justina Lai
#5707

From: Robles, Raymond C [mailto:raymond.c.robles at intel.com]
Sent: Tuesday, June 20, 2017 5:37 AM
To: Justina Lai <justina_lai at phison.com<mailto:justina_lai at phison.com>>
Cc: nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: RE: Compatibility issue with 1.5 version nvme driver

Hi Justina,

Yes, we did receive the email and have been emailing you. I notice that your emails keep getting bounced off the reflector list and I have to approve them. Could you please officially subscribe to the reflector email list so that your emails are not bounced (which require me to approve and forward).

There was an additional request for you to run the normal unit tests required for all OFA patches. I'll forward that email again after you officially subscribe the email list. Did you receive that email?

Thanks...
Ray

From: nvmewin [mailto:nvmewin-bounces at lists.openfabrics.org] On Behalf Of Justina Lai
Sent: Wednesday, May 24, 2017 10:18 PM
To: nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>
Subject: Re: [nvmewin] Compatibility issue with 1.5 version nvme driver

Hi,

Have you received my mail on 5/22?
Thanks!

Best Regards,
Justina Lai
#5707

From: Justina Lai
Sent: Monday, May 22, 2017 1:14 PM
To: 'nvmewin at lists.openfabrics.org' <nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org>>
Cc: Larry Li <larry_li at phison.com<mailto:larry_li at phison.com>>
Subject: Compatibility issue with 1.5 version nvme driver

Dear Sir/Madam,

We are facing compatibility issue on our PS5007 NVMe device with recent 1.5 version nvme driver.
Please check below issue description.

Phison PS5007 supports 7 IO queues, and if we use PS5007 with OFA driver on the platform core number>7, ex: 8-core or 12-core PC, we will see PC hang up and cannot enter OS like below:
[cid:image001.jpg at 01D2EF62.C77E6AB0]

After debugging on our side, we found the fail is caused by below flow:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
..........
1.      SntiTranslateModeSense()

..........
2.IoCompletionRoutine()
    if (pDpc != NULL) {
        ASSERT(pAE->ntldrDump == FALSE);
        if (pAE->MultipleCoresToSingleQueueFlag) {
            StorPortAcquireSpinLock(pAE, StartIoLock, NULL, &StartLockHandle); ----------------------> execute lock
        } else {
            StorPortAcquireSpinLock(pAE, DpcLock, pDpc, &DpcLockhandle);
        }
    }
.........
callStorportNotification = pSrbExtension->pNvmeCompletionRoutine(pAE, (PVOID)pSrbExtension)
                            && (pSrbExtension->pSrb != NULL);
.......
3.SntiTranslateModeSenseResponse()

.......
case MODE_SENSE_RETURN_ALL:
.......
if (supportsVwc == TRUE) {

                                                          pSrbExt->pNvmeCompletionRoutine = SntiCompletionCallbackRoutine;

                                                          /* Finally, make sure we issue the GET FEATURES command */
                                                          SntiBuildGetFeaturesCmd(pSrbExt, VOLATILE_WRITE_CACHE);

                                                          ioStarted = ProcessIo(pSrbExt->pNvmeDevExt,
                                                                   pSrbExt,
                                                                   NVME_QUEUE_TYPE_ADMIN,
                                                                   TRUE);

4.ProcessIo()
        ........
        if (AcquireLock == TRUE) {
            StorPortAcquireSpinLock(pAdapterExtension,
                                 StartIoLock,
                                 NULL,
                                 &hStartIoLock);  ------------------------> double execute lock and cause (Assertion failure - code c0000420)

        }
---------------------------------------------------------------------------------------------------------------------------------------------------------------------

Since many users are now using our PS5007 device with 8-core or 12-core PC, they are facing this issue right now. Could you please help to modify driver to solve this problem asap?

Any unclear point, please kindly let us know.
Thank you very much for the help!

Best Regards,
Justina Lai
#5707

This message and any attachments are confidential and may be legally privileged. Any unauthorized review, use or distribution by anyone other than the intended recipient is strictly prohibited. If you are not the intended recipient, please immediately notify the sender, completely delete the message and any attachments, and destroy all copies. Your cooperation will be highly appreciated.

This message and any attachments are confidential and may be legally privileged. Any unauthorized review, use or distribution by anyone other than the intended recipient is strictly prohibited. If you are not the intended recipient, please immediately notify the sender, completely delete the message and any attachments, and destroy all copies. Your cooperation will be highly appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20170627/15cbb228/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 11633 bytes
Desc: image001.jpg
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20170627/15cbb228/attachment.jpg>