<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:x =
"urn:schemas-microsoft-com:office:excel" xmlns:p =
"urn:schemas-microsoft-com:office:powerpoint" xmlns:a =
"urn:schemas-microsoft-com:office:access" xmlns:dt =
"uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s =
"uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs =
"urn:schemas-microsoft-com:rowset" xmlns:z = "#RowsetSchema" xmlns:b =
"urn:schemas-microsoft-com:office:publisher" xmlns:ss =
"urn:schemas-microsoft-com:office:spreadsheet" xmlns:oa =
"urn:schemas-microsoft-com:office:activation" xmlns:html =
"http://www.w3.org/TR/REC-html40" xmlns:q =
"http://schemas.xmlsoap.org/soap/envelope/" XMLNS:D = "DAV:" xmlns:x2 =
"http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ois =
"http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir =
"http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds =
"http://www.w3.org/2000/09/xmldsig#" xmlns:dsp =
"http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc =
"http://schemas.microsoft.com/data/udc" xmlns:xsd =
"http://www.w3.org/2001/XMLSchema" xmlns:sps =
"http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi =
"http://www.w3.org/2001/XMLSchema-instance" xmlns:udcxf =
"http://schemas.microsoft.com/data/udc/xmlfile" xmlns:wf =
"http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:mver =
"http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m =
"http://schemas.microsoft.com/office/2004/12/omml" xmlns:ex12t =
"http://schemas.microsoft.com/exchange/services/2006/types"><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.2963" name=GENERATOR><!--[if !mso]>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Cambria Math;
}
@font-face {
font-family: Calibri;
}
@font-face {
font-family: Tahoma;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.0in 1.0in 1.0in; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman","serif"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman","serif"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman","serif"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline; mso-style-priority: 99
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline; mso-style-priority: 99
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline; mso-style-priority: 99
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline; mso-style-priority: 99
}
P {
FONT-SIZE: 12pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; FONT-FAMILY: "Times New Roman","serif"; mso-style-priority: 99; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto
}
SPAN.EmailStyle18 {
COLOR: #1f497d; FONT-FAMILY: "Calibri","sans-serif"; mso-style-type: personal
}
SPAN.EmailStyle19 {
COLOR: #1f497d; FONT-FAMILY: "Calibri","sans-serif"; mso-style-type: personal-reply
}
.MsoChpDefault {
FONT-SIZE: 10pt; mso-style-type: export-only
}
DIV.Section1 {
page: Section1
}
</STYLE>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV><SPAN class=813440608-31012007><FONT face=Arial color=#0000ff size=2>As far
as I checked with Leonid the SMP is working in the HCA.</FONT></SPAN></DIV>
<DIV><SPAN class=813440608-31012007><FONT face=Arial color=#0000ff size=2>Still
its not compliance as it does not check the m-key.</FONT></SPAN></DIV>
<DIV><SPAN class=813440608-31012007><FONT face=Arial color=#0000ff size=2>I will
work to implement it move it to the IBAL.</FONT></SPAN></DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> Fab Tillier
[mailto:ftillier@windows.microsoft.com] <BR><B>Sent:</B> Wednesday, January
31, 2007 6:55 AM<BR><B>To:</B> Yossi Leybovich;
openib-windows@openib.org<BR><B>Subject:</B> RE: [Openib-windows] [ANNOUNCE]
Build 1.0.0.566 posted<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">You
should ideally be able to cache PKey and GID tables. The HCA driver’s
SMP cache is setup to do that, but I don’t remember why I didn’t set it up to
actually do it.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">Note
however that the SMP cache in the HCA driver is still invoked from the passive
level thread, so it doesn’t quite solve the problem. I don’t know if the
cache should be in IBAL. If implemented in the HCA driver, I think using
a passive thread (or even just a work item could provide async local MAD
functionality even if the underlying HCA driver implementation blocks,
allowing the elimination of the local mad complexity in
IBAL.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">-Fab<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<DIV>
<DIV
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<P class=MsoNormal><B><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'">From:</SPAN></B><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'"> Yossi Leybovich
[mailto:sleybo@dev.mellanox.co.il] <BR><B>Sent:</B> Tuesday, January 30, 2007
6:47 AM<BR><B>To:</B> 'Yossi Leybovich'; Fab Tillier;
openib-windows@openib.org<BR><B>Subject:</B> RE: [Openib-windows] [ANNOUNCE]
Build 1.0.0.566 posted<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Arial','sans-serif'">Two
more things.</SPAN><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal> <o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Arial','sans-serif'">1. pls
note that even in case that the simple m-key check is good we still need to
reset m-key lease timer (which is used by HW/FW ) </SPAN><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Arial','sans-serif'">so we
still need to forward the first good packet to the
FW.</SPAN><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Arial','sans-serif'">2.
Does there any reason why we not keep GID/Pkey tables in cache ? Cant we
answer this packets from cache ?</SPAN><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal> <o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Arial','sans-serif'">Yossi
</SPAN><o:p></o:p></P></DIV>
<DIV>
<P class=MsoNormal> <o:p></o:p></P></DIV>
<BLOCKQUOTE
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; MARGIN: 5pt 0in 5pt 3.75pt; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<P class=MsoNormal><o:p> </o:p></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center>
<HR align=center width="100%" SIZE=2>
</DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'">From:</SPAN></B><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'">
openib-windows-bounces@openib.org [mailto:openib-windows-bounces@openib.org]
<B>On Behalf Of </B>Yossi Leybovich<BR><B>Sent:</B> Monday, January 29, 2007
4:29 PM<BR><B>To:</B> 'Fab Tillier';
openib-windows@openib.org<BR><B>Subject:</B> Re: [Openib-windows] [ANNOUNCE]
Build 1.0.0.566 posted</SPAN><o:p></o:p></P>
<DIV>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Arial','sans-serif'">see
my comments below.</SPAN><o:p></o:p></P></DIV>
<BLOCKQUOTE
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: medium none; PADDING-LEFT: 4pt; PADDING-BOTTOM: 0in; MARGIN: 5pt 0in 5pt 3.75pt; BORDER-LEFT: blue 1.5pt solid; PADDING-TOP: 0in; BORDER-BOTTOM: medium none">
<P class=MsoNormal><o:p> </o:p></P>
<DIV class=MsoNormal style="TEXT-ALIGN: center" align=center>
<HR align=center width="100%" SIZE=2>
</DIV>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><B><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'">From:</SPAN></B><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'">
openib-windows-bounces@openib.org
[mailto:openib-windows-bounces@openib.org] <B>On Behalf Of </B>Fab
Tillier<BR><B>Sent:</B> Thursday, January 25, 2007 7:34 PM<BR><B>To:</B>
Yossi Leybovich; openib-windows@openib.org<BR><B>Subject:</B> Re:
[Openib-windows] [ANNOUNCE] Build 1.0.0.566 posted</SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">Hi
Yossi,<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">A
question about r538:<o:p></o:p></SPAN></P>
<P><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">------------------------------------------------------------------------<BR>r538
| sleybo | 2006-11-07 08:54:25 +0200 (Tue, 07 Nov 2006) | 3
lines<o:p></o:p></SPAN></P>
<P class=MsoNormal style="MARGIN-BOTTOM: 12pt"><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">[IBAL]
Compliance tests<BR>1. pass switch_info to the HCA - compliance test
C13-026<BR>2. Not use AL cashe for node_description node_info to force
Mkey check -compliance test
C14-018<BR>------------------------------------------------------------------------</SPAN><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">Have
you tested to see what the effects of removing the cache for node
description and node info are on SM sweeps when the system is
busy?<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">I
initially added the cache for these so that the response could be issued
in the context of the CQ callback for the special QP (thus at
DISPATCH_LEVEL). Without the cache processing requires a call to the
local MAD verb, which has to be scheduled on a passive-level thread.
If the system is very busy doing I/O (i.e. lots of small packets in
Iometer over IPoIB), I have seen cases where the local MAD thread does not
run fast enough so the response time for the MAD is too long and the SM
declares the node as having failed and removes it from the fabric.
This is pretty nasty, as suddenly all IB multicast group memberships are
lost, but there’s no indication to the host that things went
awry.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">There
are two solutions for this, one is more of a temporary fix than the other
IMO. First, the temporary fix: perform the MKey check in software,
so that the MAD response for as many MADs can be generated at
DISPATCH_LEVEL from the context of the special QP’s CQ callback.
This should maintain compliance while also keeping response times for MADs
as short as possible.<BR></SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Calibri','sans-serif'">[Yossi
Leybovich] To solve the problem of denial of service I will add
simple m-key check . In any case of error (or not trivial m-key check (i.e
m-key =0) I will disable the cache and move the MAD to the
FW</SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Calibri','sans-serif'">(
I don't want to count the m key violation and of course not to add
code that generate traps).</SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Calibri','sans-serif'">This
will reduce the handling of good flow packets.</SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><BR>The
second solution is to make the local MAD verb asynchronous. The HCA
handles the command asynchronously anyway, so this is a more natural fit
given the HW design. This would mean the local MAD verb would be
called directly from the CQ callback (at DISPATCH_LEVEL), and would return
pending. When the local MAD is processed and the HCA generates the
response to the EQ, the driver could invoke a callback to indicate
completion (again at DISPATCH_LEVEL) which would send out the
response. This solution eliminates the thread scheduling issues
associated with handling local MAD requests in a passive-level
thread.<BR></SPAN><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Calibri','sans-serif'">[Yossi
Leybovich] </SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 10pt; COLOR: blue; FONT-FAMILY: 'Calibri','sans-serif'">This
will require to test our driver with async commands ( I think that Leonid
does not fully support it Leonid?) I don't think we will have the time to
do that in the short future</SPAN><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"> <o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">We
should make sure that systems aren’t susceptible to Denial of Service
attacks from someone flooding them with IPoIB traffic (which gets handled
at DISPATCH_LEVEL in IPoIB’s CQ callback). It’s bad if an
application on one host can cause another host to be removed from the
fabric – there will be no port down events, no notification to the SM when
the host is responsive again, and the host will not be able to participate
properly in the fabric until the next SM sweep.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"> <o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">-Fab</SPAN><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'"><o:p></o:p></SPAN></P></BLOCKQUOTE></BLOCKQUOTE></DIV></BLOCKQUOTE></BODY></HTML>