<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7226.0">
<TITLE>QoS RFC</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P DIR=LTR><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype">Hi All </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype">Please find the</FONT></SPAN><SPAN LANG="en-us"> <FONT COLOR="#0000FF" FACE="Palatino Linotype">attached</FONT></SPAN><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype"> RFC describing how QoS</FONT></SPAN><SPAN LANG="en-us"> <FONT COLOR="#0000FF" FACE="Palatino Linotype">policy support</FONT></SPAN><SPAN LANG="en-us"> <FONT COLOR="#0000FF" FACE="Palatino Linotype">could be implemented</FONT></SPAN><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype"> in</FONT></SPAN><SPAN LANG="en-us"> <FONT COLOR="#0000FF" FACE="Palatino Linotype">the</FONT></SPAN><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype"> OpenFabrics stack.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype">Your comments are welcome.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT COLOR="#0000FF" FACE="Palatino Linotype">Eitan</FONT></SPAN><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"></SPAN><A NAME=""><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> RFC: OpenFabrics Enhancements for QoS Support</FONT></SPAN></A></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> ===============================================</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Authors: . Eitan Zahavi <eitan@mellanox.co.il></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Date: .... May 2006.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Revision: 0.1</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Table of contents:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">1. Overview</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2. Architecture</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">3. Supported Policy</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">4. CMA functionality</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">5. IPoIB functionality</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">6. SDP functionality</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">7. SRP functionality</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">8. iSER functionality</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">9. OpenSM functionality</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">1. Overview</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">------------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Quality of Service requirements stem from the realization of I/O consolidation </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">over IB network: As multiple applications and ULPs share the same fabric, means </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">to control their use of the network resources are becoming a must. The basic </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">need is to differentiate the service levels provided to different traffic flows. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Such that a policy could be enforced and control each flow utilization of the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">fabric resources.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">IBTA specification defined several hardware features and management interfaces </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">to support QoS:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Up to 15 Virtual Lanes (VL) could carry traffic in a non-blocking manner</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Arbitration between traffic of different VL is performed by a 2 priority </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> levels weighted round robin arbiter. The arbiter is programmable with </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> a sequence of (VL, weight) pairs and maximal number of high priority credits </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> to be processed before low priority is served</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Packets carry class of service marking in the range 0 to 15 in their</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> header SL field</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Each switch can map the incoming packet by its SL to a particular output</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> VL based on programmable table VL=SL-to-VL-MAP(in-port, out-port, SL)</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* The Subnet Administrator controls each communication flow parameters</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> by providing them as a response to Path Record query</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The IB QoS features provide the means to implement a DiffServ like architecture. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">DiffServ architecture (IETF RFC2474 2475) is widely used today in highly dynamic </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">fabrics. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">This proposal provides the detailed functional definition for the various </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">software elements that are required to enable a DiffServ like architecture over </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">the OpenFabrics software stack.</FONT></SPAN></P>
<BR>
<BR>
<BR>
<BR>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2. Architecture</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">----------------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">This proposal split the QoS functionality between the SM/SA, CMA and the various </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">ULPS. We take the “chronology approach” to describe how the overall system </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">works:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2.1. The network manager (human) provides a set of rules (policy) that defines </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">how the network is being configured and how its resources are split to different </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">QoS-Levels. The policy also define how to decide which QoS-Level each </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">application or ULP or service use.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2.2. The SM analyzes the provided policy to see if it is realizable and performs </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">the necessary fabric setup. The SM may continuously monitor the policy and adapt </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">to changes in it. Part of this policy defines the default QoS-Level of each </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">partition. The SA is being enhanced to match the requested Source, Destination, </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">TClass, Service-ID (and optionally SL and priority) against the policy. So </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">clients (ULPs, programs) can obtain a policy enforced QoS. The SM is also </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">enhanced to support setting up partitions with appropriate IPoIB broadcast </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">group. This broadcast group carries its QoS attributes: TClass, SL, MTU and </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">RATE.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2.3. IPoIB is being setup. IPoIB uses the SL, MTU and RATE available on the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">multicast group which forms the broadcast group of this partition.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2.4. MPI which provides non IB based connection management should be configured </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">to run using hard coded SLs. It uses these SLs in every QP being opened.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2.5. ULPs that use CM interface (like SRP) should have their own pre-assigned </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Service-ID and use it while obtaining PathRecord for establishing their </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">connections. The SA receiving the PathRecord should match it against the policy </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">and return the appropriate PathRecord including SL, MTU, RATE and TClass. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">2.6. ULPs and programs using CMA to establish RC connection should provide the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">CMA the target IP and Service-ID. Some of the ULPs might also provide TClass </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">(E.g. for SDP sockets that are provided the TOS socket option). The CMA should </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">then use the provided Service-ID and optional TClass and pass them in the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">PathRecord request. The resulting PathRecord should be used for configuring the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">connection QP.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">PathRecord and MultiPathRecord enhancement for QoS: </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">As mentioned above the PathRecord and MultiPathRecord attributes should be </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">enhanced to carry the Service-ID which is a 64bit value. Given the existing </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">definition for these attributes we propose to use the following fields for </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Service-ID:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* For PathRecord: use the first 2 reserved fields whicg are 32bits each </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> (component masks 0x1 and 0x2). Component mask 1 should be used to refer to the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> merged Service-ID field</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* For MultiPathRecord: use 2 reserved fields: </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> 1. after the packet life (8 bits) which is component mask bit 0x10000 (17)</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> 2. the field before SDGID1 (56 bits) which is component mask bit 0x200000 (22)</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> Once merged they should be selected using component mask bit 0x10000 (17)</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">A new capability bit should describe the SM QoS support in the SA class port </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">info.</FONT></SPAN><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> This approach provides an easy migration path for existing access layer </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">and ULPs by not introducing a new attribute.</FONT></SPAN><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"></SPAN></P>
<BR>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">3. Supported Policy</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">-------------------- </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The QoS policy supported by this proposal is divided into 4 sub sections:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Node Group: a set of HCAs, Routers or Switches that share the same settings. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">A node groups might be a partition defined by the partition manager policy in </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">terms of GUIDs. Future implementations might provide support for NodeDescription </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">based definition of node groups.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Fabric Setup: </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Defines how the SL2VL and VLArb tables should be setup. This policy definition </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">assumes the computation of target behavior should be performed outside of </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">OpenSM.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* QoS-Levels Definition:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">This section defines the possible sets of parameters for QoS that a client might </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">be mapped to. Each set holds: SL and optionally: Max MTU, Max Rate, Path Bits </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">(in case LMC > 0 is used for QoS) and TClass.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Matching Rules:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">A list of rules that match an incoming PathRecord request to a QoS-Level. The </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">rules are processed in order such as the first match is applied. Each rule is </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">built out of set of match expressions which should all match for the rule to </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">apply. The matching expressions are defined for the following fields</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">** SRC and DST to lists of node groups</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">** Service-ID to a list of Service-ID or Service-ID ranges</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">** TClass to a list of TClass values or ranges</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">XML style syntax is provided for the policy file. However, a strict BNF format </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">(provided in section 8) should be used for parsing it.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"><?xml version="1.0" encoding="ISO-8859-1"?></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"><qos-policy></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- Port Groups define sets of ports to be used later in the settings --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-groups></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- using port GUIDs --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-group> <name>Storage</name> <use>our SRP storage targets</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-guid>0x1000000000000001</port-guid></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-guid>0x1000000000000002</port-guid></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </port-group></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- using names obtained by concatenation of first 2 words of NodeDescription</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> and port number --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-group> <name>Virtual Servers</name> <use>node desc and IB port #</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-name>vs1/HCA-1/P1</port-name></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-name>vs3/HCA-1/P1</port-name></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-name>vs3/HCA-2/P1</port-name></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </port-group></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- using partitions defined in the partition policy --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-group> <name>Partition 1</name> <use>default settings</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <partition>Part1</partition> </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </port-group></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- using node types HCA|ROUTER|SWITCH --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <port-group> <name>Routers</name> <use>all routers</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <node-type>ROUTER</node-type> </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </port-group> </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </port-groups></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos-setup></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- define all types of SL2VL tables always have 16 VL entries --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl2vl-tables></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- scope defines the exact devices and in/out ports the tables apply to</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> if the same port is matching several rules the last one applies --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl2vl-scope> <group>Part1</group> <from>*</from> <to>*</to> </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl2vl-table>0,1,2,3,4,5,6,7,8,9,10,11,12,13,14</sl2vl-table></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </sl2vl-scope></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- “across” means the port just connected to the given group, </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> also the link across port 1 is probably supporting only 2 VLs --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl2vl-scope> <across>Storage</across> <from>*</from> <to>1</to></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl2vl-table>0,1,1,1,1,1,1,1,1,1,1,1,1,1,1</sl2vl-table></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </sl2vl-scope></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl2vl-tables></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- define all types of VLArb tables. The length of the tables should </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> match the physically supported tables by their target ports --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <vlarb-tables></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- scope defines the exact ports the VLArb tables apply to --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <vlarb-scope> <group>Storage</group> <to>*</to></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- VLArb table holds VL and weight pairs --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <vlarb-high>0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1</vlarb-high></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <vlarb-low>8:255,9:127,10:63,11:31,12:15,13:7,14:3</vlarb-low></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <vl-high-limit>10</vl-high-limit></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </vlarb-scope></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </vlarb-tables></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos-setup></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"><qos-levels></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- the first one is just setting SL --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos-level> <sn>1</sn> <use>for the lowest priority comm</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl>16</sl></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos-level></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- the second sets SL and TClass --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos-level> <sn>2</sn> <use>low latency best bandwidth</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl>0</sl> <tclass>7</tclass></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos-level></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!-- the whole set: SL, TClass, MTU-Limit, Rate-Limit, Path-Bits --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos-level> <sn>3</sn> <use>just an example</use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <sl>0</sl> <tclass>32</tclass> <mtu_limit>1</mtl_limit> </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <rate_limit>1</rate_limit></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos-level></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos-levels></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos_match_rules></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- matching by single criteria: tclass (list of values and ranges) --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos_match_rule> <sn>1</sn> <use>low latency by tclass 7-9 or 11></use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <tclass>7-9,11</tclass> <match-level>1</match-level></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos_match_rule></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <!—- show matching by destination group AND service-ids --></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <qos_match_rule> <sn>2</sn> <use>Storage targets connection></use></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <destination>Storage</destination> <service>22,4719</service></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> <match-level>3</match-level></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos_match_rule></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> </qos_match_rules></FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"></qos-policy></FONT></SPAN></P>
<BR>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">4. IPoIB</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">---------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">IPoIB already query the SA for its broadcast group information. The additional </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">functionality required is for IPoIB to provide the broadcast group SL, MTU, RATE </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">and TClass in every following PathRecord query performed when a new UDAV is </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">needed by IPoIB. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">We could assign a special Service-ID for IPoIB use but since all communication </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">on the same IPoIB interface shares the same QoS-Level without the ability to </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">differentiate it by target service we can ignore it for simplicity.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">5. CMA features</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">----------------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The CMA interface supports Service-ID through the notion of port space as a </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">prefixes to the port_num which is part of the sockaddr provided to </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">rdma_resolve_add(). What is missing is the explicit request for a TClass that </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">should allow the ULP (like SDP) to propagate a specific request for a class of </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">service. A mechanism for providing the TClass is available in the IPv6 address, </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">so we could use that address field. Another option is to implement a special </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">connection options API for CMA.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Missing functionality by CMA is the usage of the provided TClass and Service-ID </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">in the sent PathRecord. When a response is obtained it is an existing </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">requirement for the CMA to use the PathRecord from the response in setting up </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">the QP address vector.</FONT></SPAN></P>
<BR>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">6. SDP</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">-------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">SDP uses CMA for building its connections. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The Service-ID for SDP is 0x000000000001PPPP, where PPPP are 4 hex digits </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">holding the remote TCP/IP Port Number to connect to.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">SDP might be provided with SO_PRIORITY socket option. In that case the value </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">provided should be sent to the CMA as the TClass option of that connection. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">7. SRP</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">-------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">Current SRP implementation uses its own CM callbacks (not CMA). So SRP should </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">fill in the Service-ID in the PathRecord by itself and use that information in </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">setting up the QP. The T10 SRP standard defines the SRP Service-ID to be defined </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">by the SRP target I/O Controller (but they should also comply with IBTA Service-</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">ID rules). Anyway, the Service-ID is reported by the I/O Controller in the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">ServiceEntries DMA attribute and should be used in the PathRecord if the SA </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">reports its ability to handle QoS PathRecords.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">8. iSER</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">--------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">iSER uses CMA and thus should be very close to SDP. The Service-ID for iSER </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">should be TBD.</FONT></SPAN></P>
<BR>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">9. OpenSM features</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">-------------------</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The QoS related functionality to be provided by OpenSM can be split into two </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">main parts:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">3.1. Fabric Setup</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">During fabric initialization the SM should parse the policy and apply its </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">settings to the discovered fabric elements. The following actions should be </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">performed:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Parsing of policy</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Node Group identification. Warning should be provided for each node not </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> specified but found.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* SL2VL settings validation should be checked:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> + A warning will be provided if there are no matching targets for the SL2VL </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> setting statement. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> + An error message will be printed to the log file if an invalid setting is </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> found. A setting is invalid if it refers to:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> - Non existing port numbers of the target devices</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> - Unsupported VLs for the target device. In the later case the map to non</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> existing VLs should be replaced to VL15 i.e. packets will be dropped.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* SL2VL setting is to be performed</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* VL Arbitration table settings should be validated according to the following </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> rules:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> + A warning will be provided if there are no matching targets for the setting </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> statement</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> + An error will be provided if the port number exceeds the target ports</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> + An error will be generated if the table length exceeds device capabilities</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> + An warning will be generated if the table quote a VL that is not supported </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> by the target device</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* VL Arbitration tables will be set on the appropriate targets</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">3.2. PathRecord query handling:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">OpenSM should be able to enforce the provided policy on client request.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The overall flow for such requests is: first the request is matched against the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">defined match rules such that the target QoS-Level definition is found. Given </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">the QoS-Level a path(s) search is performed with the given restrictions imposed </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">by that level. The following two sections describe these steps.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">One issue not standardized by the IBTA is how Service-ID is carried in the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">PathRecord and MultiPathRecord attributes. There are basically two options:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">a. Replace the SM-Key field by the Service-ID. In that case no component mask</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> bit will be assigned to it. Such that if the field is zero we should treat it </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> as if the component mask bit is clear. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">b. Encode it into spare fields. For PathRecord the first two fields are reserved </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> and are 64 bit when combined. The first component mask bit maps to the first </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> reserved field and should be used for Service-ID masking. For MultiPathRecord </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> attribute there are no adjacent reserve fields that makes a 64 bit field. So </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> the reserve field following the packet-lifetime (8 bits) combined with the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> reserved field DGIDCount (56 bits) can make the Service-ID. In this case also </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> the first reserve field component mask bit should be used as the Service-ID </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> component mask bit.</FONT></SPAN></P>
<BR>
<BR>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">3.2.1. Matching rule search:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">A rule is “matching” a PathRecord request using the following criteria:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Matching rules provide values in a list of either single value, or range of </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> values. A PathRecord field is “matching” the rule field if it is explicitly </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> noted in the list of values or is one of the values covered by a range </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> included in the field values list.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Only PathRecord fields that have their component mask bit set should be</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> compared. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* For a rule to be “matching” a PathRecord request all the rule fields should be </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> “matching” their PathRecord fields. Such that a PathRecord request that does </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> not have a component mask field set for one of the rule defined fields – can </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> not match that rule.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* A PathRecord request that have a component mask bit set for one of the fields </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> that is not defined by the rule – can match the rule. </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The algorithm to be used for searching for a rule match might be as simple as a </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">sequential search through all rules or enhanced for better performance. The </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">semantics of every rule field and its matching PathRecord field are described </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">below:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Source: the SGID or SLID should be part of this group</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Destination: the DGID or DLID should be part of this group</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Service-ID: check if the requested Service-ID (available in the PathRecord old </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> SM-Key field) is matching any of this rule Service-IDs</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* TClass: check if the PathRecord TClass field is matching</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">3.2.2 PathRecord response generation:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The QoS-Level pointed by the first rule that matches the PathRecord request </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">should be used for obtaining the response SL, MTU-Limit, RATE-Limit, Path-Bits </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">and TClass. A default QoS-Level should be used if no rule is matching the query.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">The efficient algorithm for finding paths that meet the QoS-Level criteria is </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">beyond the scope of this RFC and left for the implementer to provide. However </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">the criteria by which the paths match the QoS-Level are described below:</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* SL: The paths found should all use the given SL. For that sake PathRecord </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> algorithm should traverse the path from source to destination only through </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> ports that carry a valid VL (not VL15) by the SL2VL map (should consider input </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> and output ports and SL). </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* MTU-Limit: The resulting paths MTU should not exceed the given MTU-Limit</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Rate-Limit: The resulting paths RATE should not exceed the given RATE-Limit </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> (rate limit is given in units of link BW = Width*Speed according to IBTA </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> Specification Vol-1 table-205 p-901 l-24).</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* Path-Bits: define the target LID lowest bits (number of bits defined by the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> target port PortInfo.LMC field). The path should traverse the LFT using the </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> target port LID with the path-bits set.</FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New">* TClass: should be returned in the result PathRecord. When routing is going to </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> be supported by OpenSM – we might use this field in selecting the target </FONT></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"><FONT SIZE=2 FACE="Courier New"> router too in a TBD way.</FONT></SPAN></P>
<BR>
<BR>
<P DIR=LTR><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"></SPAN><SPAN LANG="en-us"></SPAN></P>
<P DIR=LTR><SPAN LANG="en-us"></SPAN></P>
</BODY>
</HTML>