<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=US-ASCII">
<meta name=ProgId content=Word.Document>
<meta name=Generator content="Microsoft Word 10">
<meta name=Originator content="Microsoft Word 10">
<link rel=File-List href="cid:filelist.xml@01C5B3AA.A45A4A00">
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="stockticker"/>
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:DoNotRelyOnCSS/>
</o:OfficeDocumentSettings>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:WordDocument>
<w:SpellingState>Clean</w:SpellingState>
<w:GrammarState>Clean</w:GrammarState>
<w:DocumentKind>DocumentEmail</w:DocumentKind>
<w:EnvelopeVis/>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
</w:Compatibility>
<w:BrowserLevel>MicrosoftInternetExplorer4</w:BrowserLevel>
</w:WordDocument>
</xml><![endif]--><!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-parent:"";
margin:0in;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman";
mso-fareast-font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;
text-underline:single;}
span.EmailStyle17
{mso-style-type:personal-compose;
mso-style-noshow:yes;
mso-ansi-font-size:10.0pt;
mso-bidi-font-size:10.0pt;
font-family:Arial;
mso-ascii-font-family:Arial;
mso-hansi-font-family:Arial;
mso-bidi-font-family:Arial;
color:windowtext;}
span.SpellE
{mso-style-name:"";
mso-spl-e:yes;}
span.GramE
{mso-style-name:"";
mso-gram-e:yes;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;
mso-header-margin:.5in;
mso-footer-margin:.5in;
mso-paper-source:0;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:1284386956;
mso-list-type:hybrid;
mso-list-template-ids:-2052581154 1453371672 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
-->
</style>
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:"Times New Roman";}
</style>
<![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple style='tab-interval:.5in'>
<div class=Section1>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>In
a previous mail Fab asked the following questions:<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>What
functionality should <span class=SpellE>IPoIB</span> provide?<span
style='mso-spacerun:yes'> </span>Should it provide?<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>1.
IP to path<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>2.
Ethernet MAC to path<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>3.
IP to GID<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>4.
Ethernet MAC to GID<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>5.
Get all locally assigned IP addresses<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>6.
Path validation (e.g. given an IP pair and a path, return validity of path)<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-layout-grid-align:none;text-autospace:none'><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>WSD would care most about 1, 5, and 6.<span
style='mso-spacerun:yes'> </span><span class=GramE>DAPL probably only
cares about 1 and 6.</span> What about SDP?<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>Before I'll try to answer the questions I'll
say that I assume that each adapter has one <span class=SpellE>mac</span> address
and that it can have one or many IP's. This is not always the case, but
this is the very big majority of the cases.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>The questions above are not very clear since they
don't really say if this is remote IP or local <span class=SpellE><span
class=GramE>ip</span></span><span class=GramE>, that</span> should be used.
Still, I'll try to explain the way I see things.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>On </span></font><st1:stockticker><font size=2
face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>SDP</span></font></st1:stockticker><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>
and WSD (I don't know about DAPL) one gets the remote IP that he has to
connect to it. He might get the local IP. If he doesn't get the locale
IP, he should be able to find it. Windows supplies the <span class=SpellE>GetBestRoute</span>
function that does exactly this. This function exists in user mode, I'm
not sure if it exists in kernel mode, and in any case this is not something
that the IPOIB should know about. From here I assume that we have both IP's.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>Given the locale IP and the remote IP one should be
able to find both ports </span></font><st1:stockticker><font size=2
face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>GID</span></font></st1:stockticker><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>S.
This is something that the IPOIB should be able to supply.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>Let's look at the problem of converting the locale
IP to a port's GID (and probably also the HCA's GID). This
information should be known to the I</span></font><st1:stockticker><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>POI</span></font></st1:stockticker><font
size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>B,
and we can make a query that gets the answer. In appendix <span class=GramE>a</span>
I explain how one can get this information even without using the IPOIB module.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>We will have to decide if we want to take the way
from the appendix or create more changes to IPOIB. In any case, we have to note
that this information changes. No matter how we do it, this makes the problem
much more complicated (if you are pessimistic and much more interesting if you
are optimistic). <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>The second problem is getting the remote </span></font><st1:stockticker><font
size=2 face=Arial><span style='font-size:10.0pt;font-family:Arial'>GID</span></font></st1:stockticker><font
size=2 face=Arial><span style='font-size:10.0pt;font-family:Arial'> from the
remote IP. This information is only known to the IPOIB module. After doing an <span
class=SpellE><span class=GramE>arp</span></span> (can be done easily from user
mode and we should have the IP translated to the remote MAC). To my
understanding, all information should already be in the IPOIB driver as an "<span
class=SpellE>endpt</span>" (the function __<span class=SpellE>endpt_mgr_ref</span>
should return it immediately). <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>As for translating the two <span class=SpellE>gids</span> to
a path: This can be easily done by each ULP and is not something special to the
IPOIB driver. Still each ULP has to do it, so it might be a good idea to also do
the path lookup. To my filling it makes things somewhat complicated, and we can
avoid it, but as I said, it can be done. This also <span class=GramE>rises</span>
the question of doing it sync or non sync (which will make things even more
complicated).<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Path validation - I'm not sure that I understand
the question, if I have a path to validate, I can simply ask for the new path
to be created and compare them, or is there anything else in this?<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>So to summarize: <o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1;
tab-stops:list .5in'><![if !supportLists]><font size=2 face=Arial><span
style='font-size:10.0pt;font-family:Arial;mso-fareast-font-family:Arial'><span
style='mso-list:Ignore'>1)<font size=1 face="Times New Roman"><span
style='font:7.0pt "Times New Roman"'> </span></font></span></span></font><![endif]><span
dir=LTR><font size=2 face=Arial><span style='font-size:10.0pt;font-family:Arial'>Get
all locale <span class=SpellE>ip</span> addresses, <span class=GramE>We</span>
can do it or not do it, I recommend to do it in the IPOIB. As the information
changes, it would be better to create some notification mechanism, I suggest
not implementing this mechanism at start.<o:p></o:p></span></font></span></p>
<p class=MsoNormal style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1;
tab-stops:list .5in'><![if !supportLists]><font size=2 face=Arial><span
style='font-size:10.0pt;font-family:Arial;mso-fareast-font-family:Arial'><span
style='mso-list:Ignore'>2)<font size=1 face="Times New Roman"><span
style='font:7.0pt "Times New Roman"'> </span></font></span></span></font><![endif]><span
dir=LTR><font size=2 face=Arial><span style='font-size:10.0pt;font-family:Arial'>Get
remote IP - we must implement.<o:p></o:p></span></font></span></p>
<p class=MsoNormal style='margin-left:.5in;text-indent:-.25in;mso-list:l0 level1 lfo1;
tab-stops:list .5in'><![if !supportLists]><font size=2 face=Arial><span
style='font-size:10.0pt;font-family:Arial;mso-fareast-font-family:Arial'><span
style='mso-list:Ignore'>3)<font size=1 face="Times New Roman"><span
style='font:7.0pt "Times New Roman"'> </span></font></span></span></font><![endif]><span
dir=LTR><font size=2 face=Arial><span style='font-size:10.0pt;font-family:Arial'>Get/verify
path we can implement but I believe that we shouldn't do it.<o:p></o:p></span></font></span></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Do you agree? <span class=GramE>Any comments?</span><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Appendix A - How to get all the locale IP addresses (and
even to map them to InfiniBand ports): <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Windows provides the function <span class=SpellE>GetAdaptersInfo</span>
which gives all IP adapters as well as their locale IP. For each adapter there
is a "name" that is actually a <span class=SpellE>guid</span> that uniquely
describes the adapter. In the registry there is a key named: HKEY_LOCAL_MACHINE\SYSTEM\<span
class=SpellE>CurrentControlSet</span>\Control\Network<span class=GramE>\{</span>4D36E972-E325-11CE-BFC1-08002BE10318}
that has an entry for each adapter. For each of the adapters there is a sub key
call Connection. Under this there is a value that is called <span class=SpellE>PnpInstanceID</span>
this value starts with IBA\IPOIB for InfiniBand devices, so we know that this
is "our" card. Next there is the <span class=SpellE>guid</span> of
the port. <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Please note that since adapters (and more likely IP
addresses) change with time, it is not that simple to cache the information in
user mode.<o:p></o:p></span></font></p>
</div>
</body>
</html>