<HTML xmlns:v = "urn:schemas-microsoft-com:vml" xmlns:o = "urn:schemas-microsoft-com:office:office" xmlns:w = "urn:schemas-microsoft-com:office:word" xmlns:m = "http://schemas.microsoft.com/office/2004/12/omml" xmlns:ns0 = ""><HEAD><TITLE>Samsung Enterprise Portal mySingle</TITLE>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<STYLE id=mysingle_style type=text/css>P {
MARGIN-TOP: 5px; FONT-FAMILY: Arial, arial; MARGIN-BOTTOM: 5px; FONT-SIZE: 9pt
}
TD {
MARGIN-TOP: 5px; FONT-FAMILY: Arial, arial; MARGIN-BOTTOM: 5px; FONT-SIZE: 9pt
}
LI {
MARGIN-TOP: 5px; FONT-FAMILY: Arial, arial; MARGIN-BOTTOM: 5px; FONT-SIZE: 9pt
}
BODY {
LINE-HEIGHT: 1.4; MARGIN: 10px; FONT-FAMILY: Arial, arial; FONT-SIZE: 9pt
}
</STYLE>
<META name=GENERATOR content=ActiveSquare></HEAD>
<BODY>
<P>Hi Kwok,</P>
<P> </P>
<P>Its Windows-7 32 bit system and but I have n't seen the same issue on Windows 64-bit system. </P>
<P> </P>
<P>Regards</P>
<P>Santosh</P>
<P> </P>
<P> </P>
<P> </P>
<P>------- <B>Original Message</B> -------</P>
<P><B>Sender</B> : Kong, Kwok<Kwok.Kong@idt.com></P>
<P><B>Date</B> : Apr 02, 2013 06:03 (GMT+09:00)</P>
<P><B>Title</B> : RE: RE: RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list</P>
<P> </P>
<META name=Generator content=ActiveSquare>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<STYLE>@font-face {
font-family: Cambria Math;
}
@font-face {
font-family: Calibri;
}
@font-face {
font-family: Tahoma;
}
@page WordSection1 {size: 8.5in 11.0in; margin: 1.0in 1.0in 1.0in 1.0in; }
P.MsoNormal {
FONT-FAMILY: "Times New Roman","serif"; MARGIN-LEFT: 0in; FONT-SIZE: 12pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-believe-normal-left: yes
}
LI.MsoNormal {
FONT-FAMILY: "Times New Roman","serif"; MARGIN-LEFT: 0in; FONT-SIZE: 12pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-believe-normal-left: yes
}
DIV.MsoNormal {
FONT-FAMILY: "Times New Roman","serif"; MARGIN-LEFT: 0in; FONT-SIZE: 12pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-believe-normal-left: yes
}
A:link {
COLOR: blue; TEXT-DECORATION: underline; mso-style-priority: 99
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline; mso-style-priority: 99
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline; mso-style-priority: 99
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline; mso-style-priority: 99
}
P {
MARGIN: 3.75pt 0in; FONT-FAMILY: "Arial","sans-serif"; FONT-SIZE: 9pt; mso-style-priority: 99
}
P.MsoAcetate {
FONT-FAMILY: "Tahoma","sans-serif"; MARGIN-LEFT: 0in; FONT-SIZE: 8pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 99; mso-style-link: "Balloon Text Char"
}
LI.MsoAcetate {
FONT-FAMILY: "Tahoma","sans-serif"; MARGIN-LEFT: 0in; FONT-SIZE: 8pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 99; mso-style-link: "Balloon Text Char"
}
DIV.MsoAcetate {
FONT-FAMILY: "Tahoma","sans-serif"; MARGIN-LEFT: 0in; FONT-SIZE: 8pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 99; mso-style-link: "Balloon Text Char"
}
P.MsoListParagraph {
FONT-FAMILY: "Times New Roman","serif"; MARGIN-LEFT: 0.5in; FONT-SIZE: 12pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 34
}
LI.MsoListParagraph {
FONT-FAMILY: "Times New Roman","serif"; MARGIN-LEFT: 0.5in; FONT-SIZE: 12pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 34
}
DIV.MsoListParagraph {
FONT-FAMILY: "Times New Roman","serif"; MARGIN-LEFT: 0.5in; FONT-SIZE: 12pt; MARGIN-RIGHT: 0in; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-style-priority: 34
}
SPAN.BalloonTextChar {
FONT-FAMILY: "Tahoma","sans-serif"; mso-style-priority: 99; mso-style-link: "Balloon Text"; mso-style-name: "Balloon Text Char"
}
SPAN.EmailStyle21 {
FONT-FAMILY: "Calibri","sans-serif"; COLOR: #1f497d; mso-style-type: personal
}
SPAN.EmailStyle22 {
FONT-STYLE: normal; FONT-FAMILY: "Times New Roman","serif"; COLOR: #1f497d; FONT-WEIGHT: normal; mso-style-type: personal
}
SPAN.EmailStyle23 {
FONT-FAMILY: "Calibri","sans-serif"; COLOR: #1f497d; mso-style-type: personal
}
SPAN.EmailStyle24 {
FONT-FAMILY: "Calibri","sans-serif"; COLOR: #1f497d; mso-style-type: personal-reply
}
.MsoChpDefault {
FONT-SIZE: 10pt; mso-style-type: export-only
}
DIV.WordSection1 {
page: WordSection1
}
</STYLE>
<X-BODY style="MARGIN: 7.5pt" lang=EN-US link="blue" vlink="purple">
<DIV class=WordSection1>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Santosh,<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">What version of Windows are you running ? Is it 32-bit or 64-bit ?<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Officially, the current release of the driver supports Windows 7 – 64 bits only. It does not work with 32-bit Windows.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Thanks<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-Kwok<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt"><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">From:</SPAN></B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt"> SANTOSH SINGH [mailto:santosh.s2@samsung.com] <BR><B>Sent:</B> Monday, April 01, 2013 5:06 AM<BR><B>To:</B> Kong, Kwok<BR><B>Cc:</B> nvmewin@lists.openfabrics.org<BR><B>Subject:</B> Re: RE: RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P>Hi Kwok,<o:p></o:p></P>
<P> <o:p></o:p></P>
<P>I did not modify the driver code nor introdue any bug but have seen this behaviour couple of times. Will capture the debug trace.<o:p></o:p></P>
<P> <o:p></o:p></P>
<P>Regards<o:p></o:p></P>
<P>Santosh<o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P>------- <B>Original Message</B> -------<o:p></o:p></P>
<P><B>Sender</B> : Kong, Kwok<<A href="mailto:Kwok.Kong@idt.com">Kwok.Kong@idt.com</A>><o:p></o:p></P>
<P><B>Date</B> : Mar 27, 2013 07:56 (GMT+09:00)<o:p></o:p></P>
<P><B>Title</B> : RE: RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></P>
<P> <o:p></o:p></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Santosh,<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Something is not right here. The driver pre-allocates the PRP list to store up to 32 entries per list during initialization time. PRP2 must point to a list with the following offset:<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">0x000<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">0x100<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">0x200<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">…<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">…<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">0xE00<o:p></o:p></SPAN></P>
<P style="TEXT-INDENT: -0.25in; MARGIN-RIGHT: 7.5pt" class=MsoListParagraph><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-</SPAN><SPAN style="COLOR: #1f497d; FONT-SIZE: 7pt"> </SPAN><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">0xF00<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">PRP2 cannot point to an offset 0xFF0 as in your example. <o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">You can confirm this in your driver by adding the following debugging message in the file nvmeinit.c and function NVMeInitFreeQ.<o:p></o:p></SPAN></P>
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: blue; FONT-SIZE: 10pt">It's around Line# 720 to add the printout...</SPAN><o:p></o:p></P>
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal> <o:p></o:p></P>
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: blue; FONT-SIZE: 10pt"> /* Save the address of current list for calculating next list */<BR> CurPRPList = (ULONG_PTR)pCmdInfo->pPRPList;<BR> pCmdInfo->prpListPhyAddr = StorPortGetPhysicalAddress(pAE,<BR> NULL,<BR> pCmdInfo->pPRPList,<BR> &prpListSz);<BR> </SPAN><SPAN style="FONT-FAMILY: 'Arial','sans-serif'; COLOR: red; FONT-SIZE: 10pt">StorPortDebugPrint(INFO, "NVMeInitFreeQ : Entry#%d List starts 0x%llX\n", <BR> Entry, pCmdInfo->pPRPList);</SPAN><SPAN style="COLOR: red"><o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Here is a sample output:<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#0 List starts 0xFFFFFA800D69E100<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#1 List starts 0xFFFFFA800D69E200<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#2 List starts 0xFFFFFA800D69E300<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#3 List starts 0xFFFFFA800D69E400<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#4 List starts 0xFFFFFA800D69E500<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#5 List starts 0xFFFFFA800D69E600<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#6 List starts 0xFFFFFA800D69E700<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#7 List starts 0xFFFFFA800D69E800<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#8 List starts 0xFFFFFA800D69E900<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#9 List starts 0xFFFFFA800D69EA00<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#10 List starts 0xFFFFFA800D69EB00<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#11 List starts 0xFFFFFA800D69EC00<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#12 List starts 0xFFFFFA800D69ED00<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#13 List starts 0xFFFFFA800D69EE00<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#14 List starts 0xFFFFFA800D69EF00<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#15 List starts 0xFFFFFA800D69F000<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#16 List starts 0xFFFFFA800D69F100<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#17 List starts 0xFFFFFA800D6A0100<o:p></o:p></SPAN></P>
<P style="MARGIN-BOTTOM: 5pt; MARGIN-LEFT: 7.5pt; MARGIN-RIGHT: 7.5pt; mso-margin-top-alt: 5.0pt" class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#18 List starts 0xFFFFFA800D6A0200<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">STORMINI: NVMeInitFreeQ : Entry#19 List starts 0xFFFFFA800D6A0300<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">…<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">…<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Did you modify the driver and create a bug in your driver ?<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">We have not seen this problem in our testing environment.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Thanks<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-Kwok<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt"><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">From:</SPAN></B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt"> SANTOSH SINGH [<A href="mailto:santosh.s2@samsung.com">mailto:santosh.s2@samsung.com</A>] <BR><B>Sent:</B> Tuesday, March 26, 2013 3:32 AM<BR><B>To:</B> Robles, Raymond C<BR><B>Cc:</B> Kong, Kwok; <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A>; Onufryk, Peter; Wilcox, Matthew R; <A href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</A><BR><B>Subject:</B> Re: RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Hi Ray,</SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Attached is the summary of the screen shot.</SPAN><o:p></o:p></P>
<P><IMG id=_x0000_i1025 border=0 src="cid:OV69Y1GCQMNH@namo.co.kr" width=882 height=515><o:p></o:p></P>
<P> <o:p></o:p></P>
<P>We can discuss in the next WG call.<o:p></o:p></P>
<P> <o:p></o:p></P>
<P>Regards<o:p></o:p></P>
<P>Santosh<o:p></o:p></P>
<P> <o:p></o:p></P>
<P>------- <B>Original Message</B> -------<o:p></o:p></P>
<P><B>Sender</B> : Robles, Raymond C<<A href="mailto:raymond.c.robles@intel.com">raymond.c.robles@intel.com</A>><o:p></o:p></P>
<P><B>Date</B> : Mar 22, 2013 05:17 (GMT+09:00)<o:p></o:p></P>
<P><B>Title</B> : RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></P>
<P> <o:p></o:p></P>
<P class=MsoNormal><SPAN style="COLOR: #1f497d; FONT-SIZE: 10pt">Hi Santosh,<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: #1f497d; FONT-SIZE: 10pt">I am one of the original authors of the OFA Windows NVMe driver (sorry, I’m late to this thread). What do you believe is the problem? The Windows OFA driver constructs PRP lists per the NVMe spec. We’ve run for several days with numerous data integrity testing tools (without error).<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: #1f497d; FONT-SIZE: 10pt">Do you believe that the PRP list is incorrectly constructed? Based on the screen shot you sent out, the second PRP entry in the submission queue entry points to a PRP list… it should not contain the 2<SUP>nd</SUP> PRP entry. This is per the NVMe spec.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: #1f497d; FONT-SIZE: 10pt">Thanks,<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: #1f497d; FONT-SIZE: 10pt">Ray<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="COLOR: #1f497d; FONT-SIZE: 10pt"><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">From:</SPAN></B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt"> Kong, Kwok [<A href="mailto:Kwok.Kong@idt.com">mailto:Kwok.Kong@idt.com</A>] <BR><B>Sent:</B> Thursday, March 21, 2013 11:14 AM<BR><B>To:</B> <A href="mailto:santosh.s2@samsung.com">santosh.s2@samsung.com</A><BR><B>Cc:</B> <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A>; Onufryk, Peter; Wilcox, Matthew R<BR><B>Subject:</B> RE: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Santosh,<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">The specification is very clear on how the PRP list should be constructed. If you see a problem with any driver, then it is a driver problem and not a specification problem.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Please send your question to <A href="mailto:nvmewin@lists.openfabrics.org">nvmewin@lists.openfabrics.org</A> if you believe there is a driver bug.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">What is the LBA size for your testing ? 512B or 4KB ?<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">Thanks<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt">-Kwok<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-FAMILY: 'Calibri','sans-serif'; COLOR: #1f497d; FONT-SIZE: 11pt"><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt">From:</SPAN></B><SPAN style="FONT-FAMILY: 'Tahoma','sans-serif'; FONT-SIZE: 10pt"> SANTOSH SINGH [<A href="mailto:santosh.s2@samsung.com">mailto:santosh.s2@samsung.com</A>] <BR><B>Sent:</B> Thursday, March 21, 2013 3:55 AM<BR><B>To:</B> Kong, Kwok<BR><B>Cc:</B> <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A>; Onufryk, Peter; 'Wilcox, Matthew R'<BR><B>Subject:</B> Re: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P>Sorry some how the attachement is missing. Resending.<o:p></o:p></P>
<P> <o:p></o:p></P>
<P><IMG id=_x0000_i1026 border=0 src="cid:FLSX4BRZ2165@namo.co.kr" width=1600 height=860><o:p></o:p></P>
<P> <o:p></o:p></P>
<P>Regards<o:p></o:p></P>
<P>Santosh<o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P>------- <B>Original Message</B> -------<o:p></o:p></P>
<P><B>Sender</B> : SANTOSH SINGH<<A href="mailto:santosh.s2@samsung.com">santosh.s2@samsung.com</A>> Senior Chief Engineer/SRI-Bangalore-SSD Solutions/Samsung Electronics<o:p></o:p></P>
<P><B>Date</B> : Mar 21, 2013 19:45 (GMT+09:00)<o:p></o:p></P>
<P><B>Title</B> : RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<o:p></o:p></P>
<P> <o:p></o:p></P>
<P style="MARGIN-BOTTOM: 12pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; mso-margin-top-alt: 0in" class=MsoNormal><SPAN style="FONT-FAMILY: 'Arial','sans-serif'; FONT-SIZE: 9pt">Hi Kwok,<BR><BR>I got the scenario reproduced again, while issuing the FS format command.<BR>Following are the debug details.<BR><BR>Page size 4k<BR>Data Transfer size was 16 LBA<BR>PRP1 Entry 0x3e8fd060<BR>PRP2 Entry list 0xbe166ff0<BR>Total no. of PRP entries 17<BR><BR>The 16 PRP entries should fit in single page. But the PRP2 offset<BR>0xbe166ff0(PRP2) is not correct and it has run off the page(from 0xbe166ff0<BR>to 0xbe167000) which is the next page in continuity. Attached is the<BR>snapshot of the debug window for the detailed analysis.<BR><BR><BR><BR><BR><BR><BR>Regards<BR>Santosh<BR><BR><BR>-----Original Message-----<BR>From: Kong, Kwok [<A href="mailto:Kwok.Kong@idt.com">mailto:Kwok.Kong@idt.com</A>] <BR>Sent: Thursday, March 21, 2013 1:22 AM<BR>To: Wilcox, Matthew R; <A href="mailto:santosh.s2@samsung.com">santosh.s2@samsung.com</A><BR>Cc: Onufryk, Peter; <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A><BR>Subject: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<BR><BR>Santosh,<BR><BR>Your verification on the OFA driver that it uses PRP list format as in<BR>figure 2 is incorrect.<BR><BR>The OFA driver does neither figure 1, nor figure 2.<BR><BR>By default, the max request size that the mini-port supports is 128KB. If<BR>the request size is bigger than 128KB, the port driver sends multiple 128KB<BR>requests to the OFA mini-port driver.<BR><BR>The OFA mini-port driver pre-allocates the PRP list buffers during<BR>initialization. The PRP entries (a max of 32 entries for 128KB request<BR>size) never run off the end of a page as shown in figure 1 or figure 2. All<BR>PRP entries are guaranteed to fit within a single Memory Page.<BR><BR>Thanks<BR><BR>-Kwok<BR><BR><BR><BR>-----Original Message-----<BR>From: Wilcox, Matthew R [<A href="mailto:matthew.r.wilcox@intel.com">mailto:matthew.r.wilcox@intel.com</A>] <BR>Sent: Wednesday, March 20, 2013 8:26 AM<BR>To: <A href="mailto:santosh.s2@samsung.com">santosh.s2@samsung.com</A>; Kong, Kwok<BR>Cc: Onufryk, Peter; <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A><BR>Subject: RE: RE: NVMe 1.1 : Clarification on PRP2 Entry list<BR><BR>The Linux driver does neither figure 1, nor figure 2.<BR><BR>If the number of PRP entries requires more than one page, it starts at the<BR>beginning of a page. If it requires less than a page, it may start in the<BR>middle of a page, but will never run off the end of a page as shown in<BR>figure 2.<BR><BR>I have not reviewed the OFA driver to see what it does.<BR><BR>________________________________<BR>From: SANTOSH SINGH [santosh.s2@samsung.com]<BR>Sent: March 19, 2013 8:01 PM<BR>To: Kong, Kwok; Wilcox, Matthew R<BR>Cc: Onufryk, Peter; <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A><BR>Subject: Re: RE: NVMe 1.1 : Clarification on PRP2 Entry list<BR><BR><BR>Hi Kwok, Matthew,<BR><BR><BR><BR>I verified the OFA driver too and that prepares the PRP List entry as in<BR>fig-2.<BR><BR>Any reason why both the drivers(Linux and OFA) prepares the PRP2 list like<BR>fig-2. Will it change in the later version of drivers as fig-1.<BR><BR><BR><BR>Regards<BR><BR>Santosh<BR><BR><BR><BR><BR><BR><BR><BR>------- Original Message -------<BR><BR>Sender : Kong, Kwok<BR><BR>Date : Mar 20, 2013 01:01 (GMT+09:00)<BR><BR>Title : RE: NVMe 1.1 : Clarification on PRP2 Entry list<BR><BR><BR>I believe the specification is very clear to indicate that figure 1 is<BR>correct.<BR><BR>"The last entry within a memory page, as indicated by the memory page size<BR>in the CC.MPS field, shall be a PRP List pointer if there is more than a<BR>single memory page of data to be transferred.".<BR><BR><BR>Thanks<BR><BR>-Kwok<BR><BR><BR><BR>From: Onufryk, Peter [<A href="mailto:Peter.Onufryk@idt.com">mailto:Peter.Onufryk@idt.com</A>]<BR>Sent: Tuesday, March 19, 2013 7:22 AM<BR>To: Santosh Singh; <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A><BR>Subject: RE: NVMe 1.1 : Clarification on PRP2 Entry list<BR><BR>Santosh,<BR><BR>John and I discussed this and we believe that Figure 1 is correct and that<BR>the best way to clarify this is by adding a figure showing it to the spec.<BR>This will be the first item on the agenda in this week's calls.<BR><BR>Regards,<BR>Peter<BR><BR><BR>From: Santosh Singh [<A href="mailto:santosh.s2@samsung.com">mailto:santosh.s2@samsung.com</A>]<BR>Sent: Tuesday, March 19, 2013 5:49 AM<BR>To: <A href="mailto:technical@nvmexpress.org">technical@nvmexpress.org</A><ns0:technical_x0040_nvmexpress.org><BR>Subject: NVMe 1.1 : Clarification on PRP2 Entry list<BR><BR>Hi All,<BR><BR><BR>I got the query on PRP2 , when it is entry list and not memory page<BR>aligned from a design engineer .<BR><BR>The below paragraph is from section 4.3 'Physical Region Page Entry and<BR>List' of Spec 1.1.<BR><BR>[<A href="cid:Z5JE7EUABGFC@namo.co.kr">cid:Z5JE7EUABGFC@namo.co.kr</A>]<BR><BR>PRP entry 2, when pointing to a list may also have a non-zero offset within<BR>a memory page, means that is not memory page aligned.<BR>The last entry within a memory page shall be a list pointer. There are<BR>following two understandings for this:<BR><BR>1000<BR><BR>h<BR><BR>1<BR><BR>FFFh<BR><BR>PRP entry<BR><BR>2<BR><BR>pointing to a list<BR><BR>Entry<BR><BR>1<BR><BR>Entry<BR><BR>2<BR><BR>Entry<BR><BR>3<BR><BR>Entry<BR><BR>4<BR><BR>Address of PRP list<BR><BR>5000<BR><BR>h<BR><BR>5<BR><BR>FFFh<BR><BR>Entry<BR><BR>5<BR><BR>Entry<BR><BR>6<BR><BR>Entry<BR><BR>7<BR><BR>Entry<BR><BR>8<BR><BR>1000<BR><BR>h<BR><BR>1<BR><BR>FFFh<BR><BR>PRP entry<BR><BR>2<BR><BR>pointing to a list<BR><BR>Entry<BR><BR>1<BR><BR>Entry<BR><BR>2<BR><BR>Entry<BR><BR>3<BR><BR>Entry<BR><BR>4<BR><BR>2<BR><BR>FFFh<BR><BR>Entry<BR><BR>5<BR><BR>Entry<BR><BR>6<BR><BR>Entry<BR><BR>7<BR><BR>Entry<BR><BR>8<BR><BR>Entry<BR><BR>511<BR><BR>Entry<BR><BR>510<BR><BR>Address of PRP list<BR><BR>5000<BR><BR>h<BR><BR>5<BR><BR>FFFh<BR><BR>Entry<BR><BR>512<BR><BR>Entry<BR><BR>513<BR><BR>Entry<BR><BR>514<BR><BR>Fig:1<BR><BR>Fig:2<BR><BR>Page boundary<BR><BR><BR>So just want to verify with others that as per the line in section 4.3 'A<BR>physical region page list (PRP List) is a set of PRP entries in a single<BR>page of contiguous memory'<BR>fig:2 is the correct understanding.<BR><BR><BR>Thanks & Regards<BR>Santosh<BR><BR><BR><BR><BR><BR><BR><BR><BR><BR><BR>[<A href="cid:LK7CT9SZN3WZ@namo.co.kr">cid:LK7CT9SZN3WZ@namo.co.kr</A>]<BR><BR><BR><BR><BR></ns0:technical_x0040_nvmexpress.org><o:p></o:p></SPAN></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<TABLE id=confidentialsignimg class=MsoNormalTable border=0 cellPadding=0>
<TBODY>
<TR>
<TD style="PADDING-BOTTOM: 0.75pt; PADDING-LEFT: 0.75pt; PADDING-RIGHT: 0.75pt; PADDING-TOP: 0.75pt">
<P><IMG id=_x0000_i1027 border=0 src="cid:AFQ4B6TCPYIQ@namo.co.kr" width=520 height=144><o:p></o:p></P></TD></TR></TBODY></TABLE>
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><IMG id=_x0000_i1028 border=0 src="http://ext.samsung.net/mailcheck/SeenTimeChecker?do=ba69d47c78c3acc08d47d8c18e24da0151171515984b9d550ad7d0699a0799098adfa564d3c39ac365b186a42a35dd3e259756a7cc35ba77326bbdfb2ea96a2fcf878f9a26ce15a0"><o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><IMG id=_x0000_i1029 border=0 src="http://ext.samsung.net/mailcheck/SeenTimeChecker?do=ba69d47c78c3acc0b6872ba05ecac647cb20e510c9bc8be10ad7d0699a0799098adfa564d3c39ac365b186a42a35dd3e259756a7cc35ba77326bbdfb2ea96a2fcf878f9a26ce15a0"><o:p></o:p></P>
<P> <o:p></o:p></P>
<P> <o:p></o:p></P>
<P style="MARGIN: 0in 0in 0pt" class=MsoNormal><IMG id=_x0000_i1030 border=0 src="http://ext.samsung.net/mailcheck/SeenTimeChecker?do=5f64b9fd8cb08cc2f66a4f6be00ce242c653ccac87eca576932338572397a37994dfdc2a6b2aa2df7b37768dabeed03adb9fdddda33e82cbe4a391424e62fcf6cf878f9a26ce15a0"><o:p></o:p></P></DIV></X-BODY>
<P> </P>
<P> </P><!--SP:santosh.s2--><!--santosh.s2:EP--></BODY></HTML><img src='http://ext.samsung.net/mailcheck/SeenTimeChecker?do=5f64b9fd8cb08cc2fdd126d80ad984713eb55cd9d5cabd6dc883f74d3027808b94d548315d7b78b3f676ccf8e4fcda078aa631650c9c0a6c62e1ac75b522795a07805447a154a46fcf878f9a26ce15a0' border=0 width=0 height=0 style='display:none'>