[nvmewin] NVMe 1.1 : Clarification on PRP2 Entry list
Robles, Raymond C
raymond.c.robles at intel.com
Thu Mar 21 13:17:44 PDT 2013
Hi Santosh,
I am one of the original authors of the OFA Windows NVMe driver (sorry, I'm late to this thread). What do you believe is the problem? The Windows OFA driver constructs PRP lists per the NVMe spec. We've run for several days with numerous data integrity testing tools (without error).
Do you believe that the PRP list is incorrectly constructed? Based on the screen shot you sent out, the second PRP entry in the submission queue entry points to a PRP list... it should not contain the 2nd PRP entry. This is per the NVMe spec.
Thanks,
Ray
From: Kong, Kwok [mailto:Kwok.Kong at idt.com]
Sent: Thursday, March 21, 2013 11:14 AM
To: santosh.s2 at samsung.com
Cc: technical at nvmexpress.org; Onufryk, Peter; Wilcox, Matthew R
Subject: RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list
Santosh,
The specification is very clear on how the PRP list should be constructed. If you see a problem with any driver, then it is a driver problem and not a specification problem.
Please send your question to nvmewin at lists.openfabrics.org<mailto:nvmewin at lists.openfabrics.org> if you believe there is a driver bug.
What is the LBA size for your testing ? 512B or 4KB ?
Thanks
-Kwok
From: SANTOSH SINGH [mailto:santosh.s2 at samsung.com]
Sent: Thursday, March 21, 2013 3:55 AM
To: Kong, Kwok
Cc: technical at nvmexpress.org<mailto:technical at nvmexpress.org>; Onufryk, Peter; 'Wilcox, Matthew R'
Subject: Re: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list
Sorry some how the attachement is missing. Resending.
[cid:image001.png at 01CE2635.EFEEF460]
Regards
Santosh
------- Original Message -------
Sender : SANTOSH SINGH<santosh.s2 at samsung.com<mailto:santosh.s2 at samsung.com>> Senior Chief Engineer/SRI-Bangalore-SSD Solutions/Samsung Electronics
Date : Mar 21, 2013 19:45 (GMT+09:00)
Title : RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list
Hi Kwok,
I got the scenario reproduced again, while issuing the FS format command.
Following are the debug details.
Page size 4k
Data Transfer size was 16 LBA
PRP1 Entry 0x3e8fd060
PRP2 Entry list 0xbe166ff0
Total no. of PRP entries 17
The 16 PRP entries should fit in single page. But the PRP2 offset
0xbe166ff0(PRP2) is not correct and it has run off the page(from 0xbe166ff0
to 0xbe167000) which is the next page in continuity. Attached is the
snapshot of the debug window for the detailed analysis.
Regards
Santosh
-----Original Message-----
From: Kong, Kwok [mailto:Kwok.Kong at idt.com]
Sent: Thursday, March 21, 2013 1:22 AM
To: Wilcox, Matthew R; santosh.s2 at samsung.com<mailto:santosh.s2 at samsung.com>
Cc: Onufryk, Peter; technical at nvmexpress.org<mailto:technical at nvmexpress.org>
Subject: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list
Santosh,
Your verification on the OFA driver that it uses PRP list format as in
figure 2 is incorrect.
The OFA driver does neither figure 1, nor figure 2.
By default, the max request size that the mini-port supports is 128KB. If
the request size is bigger than 128KB, the port driver sends multiple 128KB
requests to the OFA mini-port driver.
The OFA mini-port driver pre-allocates the PRP list buffers during
initialization. The PRP entries (a max of 32 entries for 128KB request
size) never run off the end of a page as shown in figure 1 or figure 2. All
PRP entries are guaranteed to fit within a single Memory Page.
Thanks
-Kwok
-----Original Message-----
From: Wilcox, Matthew R [mailto:matthew.r.wilcox at intel.com]
Sent: Wednesday, March 20, 2013 8:26 AM
To: santosh.s2 at samsung.com<mailto:santosh.s2 at samsung.com>; Kong, Kwok
Cc: Onufryk, Peter; technical at nvmexpress.org<mailto:technical at nvmexpress.org>
Subject: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list
The Linux driver does neither figure 1, nor figure 2.
If the number of PRP entries requires more than one page, it starts at the
beginning of a page. If it requires less than a page, it may start in the
middle of a page, but will never run off the end of a page as shown in
figure 2.
I have not reviewed the OFA driver to see what it does.
________________________________
From: SANTOSH SINGH [santosh.s2 at samsung.com]
Sent: March 19, 2013 8:01 PM
To: Kong, Kwok; Wilcox, Matthew R
Cc: Onufryk, Peter; technical at nvmexpress.org<mailto:technical at nvmexpress.org>
Subject: Re: RE: NVMe 1.1 : Clarification on PRP2 Entry list
Hi Kwok, Matthew,
I verified the OFA driver too and that prepares the PRP List entry as in
fig-2.
Any reason why both the drivers(Linux and OFA) prepares the PRP2 list like
fig-2. Will it change in the later version of drivers as fig-1.
Regards
Santosh
------- Original Message -------
Sender : Kong, Kwok
Date : Mar 20, 2013 01:01 (GMT+09:00)
Title : RE: NVMe 1.1 : Clarification on PRP2 Entry list
I believe the specification is very clear to indicate that figure 1 is
correct.
"The last entry within a memory page, as indicated by the memory page size
in the CC.MPS field, shall be a PRP List pointer if there is more than a
single memory page of data to be transferred.".
Thanks
-Kwok
From: Onufryk, Peter [mailto:Peter.Onufryk at idt.com]
Sent: Tuesday, March 19, 2013 7:22 AM
To: Santosh Singh; technical at nvmexpress.org<mailto:technical at nvmexpress.org>
Subject: RE: NVMe 1.1 : Clarification on PRP2 Entry list
Santosh,
John and I discussed this and we believe that Figure 1 is correct and that
the best way to clarify this is by adding a figure showing it to the spec.
This will be the first item on the agenda in this week's calls.
Regards,
Peter
From: Santosh Singh [mailto:santosh.s2 at samsung.com]
Sent: Tuesday, March 19, 2013 5:49 AM
To: technical at nvmexpress.org<mailto:technical at nvmexpress.org>
Subject: NVMe 1.1 : Clarification on PRP2 Entry list
Hi All,
I got the query on PRP2 , when it is entry list and not memory page
aligned from a design engineer .
The below paragraph is from section 4.3 'Physical Region Page Entry and
List' of Spec 1.1.
[cid:Z5JE7EUABGFC at namo.co.kr]
PRP entry 2, when pointing to a list may also have a non-zero offset within
a memory page, means that is not memory page aligned.
The last entry within a memory page shall be a list pointer. There are
following two understandings for this:
1000
h
1
FFFh
PRP entry
2
pointing to a list
Entry
1
Entry
2
Entry
3
Entry
4
Address of PRP list
5000
h
5
FFFh
Entry
5
Entry
6
Entry
7
Entry
8
1000
h
1
FFFh
PRP entry
2
pointing to a list
Entry
1
Entry
2
Entry
3
Entry
4
2
FFFh
Entry
5
Entry
6
Entry
7
Entry
8
Entry
511
Entry
510
Address of PRP list
5000
h
5
FFFh
Entry
512
Entry
513
Entry
514
Fig:1
Fig:2
Page boundary
So just want to verify with others that as per the line in section 4.3 'A
physical region page list (PRP List) is a set of PRP entries in a single
page of contiguous memory'
fig:2 is the correct understanding.
Thanks & Regards
Santosh
[cid:LK7CT9SZN3WZ at namo.co.kr]
[cid:image002.gif at 01CE2635.EFEEF460]
[http://ext.samsung.net/mailcheck/SeenTimeChecker?do=ba69d47c78c3acc08d47d8c18e24da0151171515984b9d550ad7d0699a0799098adfa564d3c39ac365b186a42a35dd3e259756a7cc35ba77326bbdfb2ea96a2fcf878f9a26ce15a0]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20130321/1fccf197/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 126969 bytes
Desc: image001.png
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20130321/1fccf197/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 14036 bytes
Desc: image002.gif
URL: <http://lists.openfabrics.org/pipermail/nvmewin/attachments/20130321/1fccf197/attachment.gif>
More information about the nvmewin
mailing list