<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word"><HEAD><TITLE>Agenda for the OFED meeting today (Jan 5, 09)</TITLE>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.2900.3429" name=GENERATOR><!--[if !mso]>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE>@font-face {
font-family: Wingdings;
}
@font-face {
font-family: MS Mincho;
}
@font-face {
font-family: Tahoma;
}
@font-face {
font-family: Comic Sans MS;
}
@font-face {
font-family: @MS Mincho;
}
@page Section1 {size: 8.5in 11.0in; margin: 1.0in 1.25in 1.0in 1.25in; }
P.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
LI.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
DIV.MsoNormal {
FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt; FONT-FAMILY: "Times New Roman"
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
P {
FONT-SIZE: 12pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in; FONT-FAMILY: "Times New Roman"; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto
}
SPAN.EmailStyle18 {
FONT-WEIGHT: normal; COLOR: maroon; FONT-STYLE: normal; FONT-FAMILY: "Comic Sans MS"; TEXT-DECORATION: none; mso-style-type: personal-reply
}
DIV.Section1 {
page: Section1
}
OL {
MARGIN-BOTTOM: 0in
}
UL {
MARGIN-BOTTOM: 0in
}
</STYLE>
</HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left><FONT
face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=925213818-05012009>There are scaling issues with SA path-record
queries. We attempted to be good citizens with Intel MPI using the
rdma_cm agent (via uDAPL) but was forced to build hard-coded RC QP
support in OFED 1.4 (uDAPL scm) to avoid the many scaling and
configuration problems that came with IPoIB requirements, ARP storms,
rdma_cm timers, and SA path record
query/caching. </SPAN></FONT></FONT></FONT></DIV><FONT face=Arial><FONT
color=#0000ff><FONT size=2><SPAN
class=925213818-05012009></SPAN></FONT></FONT></FONT></BLOCKQUOTE>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV><FONT face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=925213818-05012009>If someone wants to sign up to design and implement a
scalable SA query caching agent we would be happy to look at path record
queries again. </SPAN></FONT></FONT></FONT></DIV>
<DIV><FONT face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=925213818-05012009></SPAN></FONT></FONT></FONT> </DIV>
<DIV><FONT face=Arial><FONT color=#0000ff><FONT size=2><SPAN
class=925213818-05012009>-arlin</SPAN></FONT></FONT></FONT></DIV>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">Another
suggestion for 1.5 <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">Implementation
of SA queries for Path Records (using IBTA 1.2.1 ServiceId field) in all OFED
ULPs, especially for MPI<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">
The IBTA standard defines that the proper way to establish a connection is to
get a PathRecord from the SM/SA and use it to define all the attributes of the
communication path. <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="TEXT-INDENT: 0.5in"><FONT face="Comic Sans MS"
color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">Ideally
the IBTA CM should then be used to establish the connection and QPs as
well.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="TEXT-INDENT: 0.5in"><FONT face="Comic Sans MS"
color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal style="TEXT-INDENT: 0.5in"><FONT face="Comic Sans MS"
color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">At
present, openmpi, mvapich1 and mvapich2 do not use PathRecords, but instead
hard code attributes like the PKey, SL, etc.
<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in; TEXT-INDENT: 0.5in"><FONT
face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">In some
cases these hardcoded values can be overridden by configurable values such as
PKey and SL, but such values must be uniform across all connections and must
be provided per job (which can be error
prone/tedious).<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">
At present opensm supports PKeys and SLs, however MPI cannot easily use these
features. <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal style="MARGIN-LEFT: 0.5in; TEXT-INDENT: 0.5in"><FONT
face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">Other
features, such as lash routing, in opensm do not work properly with MPI
because the SL must be uniform across all connections, but for lash it will
vary per route.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal style="TEXT-INDENT: 0.5in"><FONT face="Comic Sans MS"
color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'">Additionally,
applications which do not use PathRecords will have difficulties with advanced
features like IB routing, partitioning, etc. All of which are available
or being worked on in opensm.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face="Comic Sans MS" color=maroon size=2><SPAN
style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Comic Sans MS'"><o:p> </o:p></SPAN></FONT></P>
<DIV><FONT face=Arial color=#0000ff
size=2></FONT> </DIV></DIV></BLOCKQUOTE></BODY></HTML>