<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:st="" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="country-region"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="place"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=PL link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>docs/nes: Add
description of iWARP Multicast Acceleration <o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>The document
describes a usage model of the iWARP Multicast Acceleration<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>Signed-off-by:
Mirek Walukiewicz <miroslaw.walukiewicz@intel.com><o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>---<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>diff --git
a/nes_release_notes.txt b/nes_release_notes.txt<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>index
6386c54..b23abf3 100644<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>---
a/nes_release_notes.txt<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+++
b/nes_release_notes.txt<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>@@ -269,5 +269,66
@@ Example mpirun command line:<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>
-mca btl_openib_flags 2<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>
/usr/mpi/gcc/openmpi-1.4.1/tests/IMB-3.2/IMB-MPI1<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+===================================<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+iWARP Multicast
Acceleration (IMA)<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+===================================<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+iWARP multicast
acceleration bases on new defined QP<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+type
IBV_QPT_RAW_ETH. Adding a IBV_QPT_RAW_ETH enables<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+a L2 traffic
kernel bypass using user-space verbs API.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+The L2 RAW_ETH
acceleration assumes that user application<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+transmits and
receives a whole L2 frame including MAC/IP/UDP/TCP headers.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+ETH RAW QP
usage:<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+first the
application creates IBV_QPT_RAW_ETH QP with associated CQ,PD,<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+completion
channels as it is performed for RDMA connection.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+Next step is
enabling L2 MAC address RX filters for directing received<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+multicasts to
the RAW_ETH QPs using ibv_attach_multicast() verb.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+From this point
the application is ready to receive and transmit<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+multicast
traffic.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+In multicast
acceleration the user application passes to ibv_post_send()<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+whole IGMP frame
including MAC header, IP header, UDP header and UDP payload.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+It is a user
responsibility to make IP fragmentaion when required payload<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+is larger than
MTU. Every fragment is a separate L2 frame to transmit.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+The
ibv_poll_cq() provides an information about the status of transmit buffer.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+On receive path
when ibv_poll_cq() provides an information<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+about received
L2 packet, the Rx buffer (previously posted by ibv_post_recv()<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+ contains a
whole L2 frame including MAC header, IP header and UDP header.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+It is a user
application responsibility to check if received packet is<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+a valid UDP
frame so the fragments must be checked and<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+checksums must
be computed.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+IMA API
description (NE020 specific) :<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+User application
must create separate CQs for RX and TX path.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+Only single sge
on tranmit is supported.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+User application
must post at least 65 rx buffers to keep RX path working.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+IMA device:<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+IMA requires
creation of the /dev/infiniband/nes_ud_sksq device to get<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+ access to
optimized IMA transmit path. The best method for creation<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+of that device
is manual addition following line to<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+/etc/udev/rules.d/90-ib.rules
file<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+after OFED
distribution installation and rebooting machine.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="nes_ud_sksq",
NAME="infiniband/%k", MODE="0644"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+As a result the
90-ib.rules should look like:<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="umad*",
NAME="infiniband/%k"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="issm*",
NAME="infiniband/%k"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="ucm*",
NAME="infiniband/%k", MODE="0666"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="uverbs*",
NAME="infiniband/%k", MODE="0666"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="ucma",
NAME="infiniband/%k", MODE="0666"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="rdma_cm",
NAME="infiniband/%k", MODE="0666"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+KERNEL=="nes_ud_sksq",
NAME="infiniband/%k", MODE="0644"<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+Keep in mind
that setting an other that 644 mode of nes_ud_sksq device<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+can cause
serious serious security issues, so it is not recommended to<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'>+change this mode
for applications.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'> <o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'> NetEffect
is a trademark of Intel Corporation in the <st1:country-region w:st="on"><st1:place
w:st="on">U.S.</st1:place></st1:country-region> and other countries.<o:p></o:p></span></font></p>
<p class=MsoNormal style='text-autospace:none'><font size=2 face="Courier New"><span
lang=EN-US style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span lang=EN-US style='font-size:
10.0pt;font-family:Arial'><o:p> </o:p></span></font></p>
</div>
</body>
</html>