<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<title>QoS RFC</title>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Palatino Linotype";
panose-1:2 4 5 2 5 5 5 3 3 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
p
{mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Eitan<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Let’s assume for the moment (because
you’re presenting this under the OpenFabrics banner) that the capability
you describe below will be hidden behind wire protocol agnostic API that can be
used by IB HCA, Ethernet RNIC, etc.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>It is not clear from the description, but
I assume the arbiter you refer to below allows for traffic classes where a) the
bandwidth of the different flows within a particular class aggregate to a fixed
bandwidth, and also b) where each flow within a class has a fixed bandwidth,
e.g. where each flow corresponds to some type of fixed rate media stream.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>A QoS API should also needs have support
for flows with large RTT (Round Trip Times). Large RTT flows typically require
the sender to space the packets equally on the wire (referred to as pacing),
with the spacing interval being dynamically computed based on the RTT. It is
sufficient in this case to be able to set a pacing attribute through the API.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Regards,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Asgeir Eiriksson<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>CTO<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Chelsio Communications<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 face=Tahoma><span style='font-size:10.0pt;
font-family:Tahoma;font-weight:bold'>From:</span></font></b><font size=2
face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'>
openib-general-bounces@openib.org [mailto:openib-general-bounces@openib.org] <b><span
style='font-weight:bold'>On Behalf Of </span></b>Eitan Zahavi<br>
<b><span style='font-weight:bold'>Sent:</span></b> Tuesday, May 30, 2006 7:41
AM<br>
<b><span style='font-weight:bold'>To:</span></b> openib-general@openib.org<br>
<b><span style='font-weight:bold'>Cc:</span></b> Nimrod Gindi; Roland Dreier<br>
<b><span style='font-weight:bold'>Subject:</span></b> [openib-general] QoS RFC</span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
<p><font size=3 color=blue face="Palatino Linotype"><span style='font-size:
12.0pt;font-family:"Palatino Linotype";color:blue'>Hi All </span></font><o:p></o:p></p>
<p><font size=3 color=blue face="Palatino Linotype"><span style='font-size:
12.0pt;font-family:"Palatino Linotype";color:blue'>Please find the</span></font>
<font color=blue face="Palatino Linotype"><span style='font-family:"Palatino Linotype";
color:blue'>attached RFC describing how QoS</span></font> <font color=blue
face="Palatino Linotype"><span style='font-family:"Palatino Linotype";
color:blue'>policy support</span></font> <font color=blue
face="Palatino Linotype"><span style='font-family:"Palatino Linotype";
color:blue'>could be implemented in</span></font> <font color=blue
face="Palatino Linotype"><span style='font-family:"Palatino Linotype";
color:blue'>the OpenFabrics stack.</span></font><o:p></o:p></p>
<p><font size=3 color=blue face="Palatino Linotype"><span style='font-size:
12.0pt;font-family:"Palatino Linotype";color:blue'>Your comments are welcome.</span></font><o:p></o:p></p>
<p><font size=3 color=blue face="Palatino Linotype"><span style='font-size:
12.0pt;font-family:"Palatino Linotype";color:blue'>Eitan</span></font><o:p></o:p></p>
<p><a name=""><font size=2 face="Courier New"><span style='font-size:10.0pt;
font-family:"Courier New"'>
RFC: OpenFabrics Enhancements for QoS Support</span></font></a><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>
===============================================</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>Authors: . Eitan Zahavi <eitan@mellanox.co.il></span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>Date: .... May 2006.</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>Revision: 0.1</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>Table of contents:</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>1. Overview</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>2. Architecture</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>3. Supported Policy</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>4. CMA functionality</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>5. IPoIB functionality</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>6. SDP functionality</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>7. SRP functionality</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>8. iSER functionality</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>9. OpenSM functionality</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>1. Overview</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>------------</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>Quality of Service requirements stem from the realization of I/O
consolidation </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>over IB network: As multiple applications and ULPs share the
same fabric, means </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>to control their use of the network resources are becoming a
must. The basic </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>need is to differentiate the service levels provided to
different traffic flows. </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>Such that a policy could be enforced and control each flow
utilization of the </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>fabric resources.</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>IBTA specification defined several hardware features and
management interfaces </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>to support QoS:</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>* Up to 15 Virtual Lanes (VL) could carry traffic in a
non-blocking manner</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>* Arbitration between traffic of different VL is performed by a
2 priority </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'> levels weighted round robin arbiter. The arbiter is
programmable with </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'> a sequence of (VL, weight) pairs and maximal number of
high priority credits </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'> to be processed before low priority is served</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>* Packets carry class of service marking in the range 0 to 15 in
their</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'> header SL field</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>* Each switch can map the incoming packet by its SL to a
particular output</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'> VL based on programmable table VL=SL-to-VL-MAP(in-port,
out-port, SL)</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>* The Subnet Administrator controls each communication flow
parameters</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'> by providing them as a response to Path Record query</span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>The IB QoS features provide the means to implement a DiffServ
like architecture. </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>DiffServ architecture (IETF RFC2474 2475) is widely used today
in highly dynamic </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>fabrics. </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>This proposal provides the detailed functional definition for
the various </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>software elements that are required to enable a DiffServ like
architecture over </span></font><o:p></o:p></p>
<p><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:
"Courier New"'>the OpenFabrics software stack.</span></font><o:p></o:p></p>
<p><font size=3 face="Times New Roman"><span style='font-size:12.0pt'><br>
<font color=navy><span style='color:navy'>< cut to end of original email></span></font><o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'><o:p> </o:p></span></font></p>
</div>
</div>
</body>
</html>