<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="PlaceType"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="PlaceName"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="place"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Courier;
panose-1:2 7 4 9 2 2 5 2 4 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Hello,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>After spending some time installing OpenIB with AMSO1100, I
have managed to get rping and krping to work. Here is a howto document describing
the steps I took in the process. This does not get opensm to work,
as I am still getting a port guid error on that front. You are welcome to
add this to any documentation area if you find it useful. I have not
written a detailed howto like this before, and I am also new to this technology,
so forgive me if there are any errors.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Cheers,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Chris Kasten<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>---------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>How to install OpenIB with an Ammasso1100 network card<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Enables rping, krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Based on the version from May 10, 2006.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>---------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Follow the Installation Cheat Sheet with a few variations
detailed below (substituting Ammasso for Mellanox):<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>https://openib.org/tiki/tiki-index.php?page=Installation+Cheat+Sheet<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>___________<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Step 3:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>I used linux kernel 2.6.16.15<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>In make menuconfig include the following as modules-<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Device Drivers -> Infiniband support -> <M>
Infiniband support<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> <M>
Infiniband userspace MAD support<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> <M>
Infiniband userspace access (verbs & cm)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> <M>
Ammasso 1100 HCA support<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> <M>
Kernel RDMA Ping Module<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>______________<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Steps 5/6:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>--------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Ignore ib_mthca on modprobe<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>______________________________<o:p></o:p></span></font></p>
<p class=MsoNormal><st1:place w:st="on"><st1:PlaceName w:st="on"><font size=2
face=Courier><span style='font-size:10.0pt;font-family:Courier'>ICS</span></font></st1:PlaceName><font
size=2 face=Courier><span style='font-size:10.0pt;font-family:Courier'> <st1:PlaceType
w:st="on">Building</st1:PlaceType></span></font></st1:place><font size=2
face=Courier><span style='font-size:10.0pt;font-family:Courier'> management
tools:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>In src/userspace/management/osm/<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> rename authors ->
AUTHORS<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>
news -> NEWS<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>
readme -> README<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> then run ./autogen.sh
&& ./configure<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>___________________________________<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Building userspace verbs libraries:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Install sysfsutils before you build these libraries.
I used sysfsutils-1.3.0<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>./configure<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>make<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>make install<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Instead of libmthca, build libamso<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>For rping, build librdmacm<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Use the standard:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>./autogen.sh<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>./configure<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>make<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>make install<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>for these libraries.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>_______________________________________________<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Don’t bother with the “Testing” step and
beyond<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Download and install:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>http://www.opengridcomputing.com/downloads/ogc_amso_kit_20060308.tgz.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>On one of my machines, the following was not added to
/etc/modprobe.conf.dist:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>(as one line)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>install ib_core for i in ib_core ib_mad ib_cm ib_sa ib_ucm
ib_umad ib_uverbs; do /sbin/modprobe<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'> --ignore-install $i; done<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Make sure this is in /etc/modprobe.conf.dist so the
modules will load automatically at boot<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Check to see that the following modules are loaded:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ib_uverbs - - <-uverbs adds /dev/infiniband/uverbs0<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>rdma_cm<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>iw_cm<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ib_addr<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ib_cm<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ib_local_sa<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ib_sa<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ib_core<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>iw_c2 - - - - <-amso<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----These two didn't load automatically for me------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>rdma_krping - <-krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>rdma_ucm- - - <-rping adds /dev/infiniband/rdma_cm<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Start up interfaces<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>After a fresh reboot, check which modules are
loaded. I had to add these two.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>modprobe rdma_krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>modprobe rdma_ucm<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Machine 1: ifconfig iw0 192.168.69.149 up<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>(server) ifconfig eth0 192.168.68.149 up<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Machine 2: ifconfig iw1 192.168.69.148 up<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>(client) ifconfig eth1 192.168.68.148 up<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----------------------------------------------------------------------------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Testing:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>______<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>------<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>On server: /bin/echo
server,port=9999,addr=192.168.69.149,validate > /proc/krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>On client: /bin/echo
client,port=9999,addr=192.168.69.149,validate > /proc/krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>(addr = server iw addr)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>To verify: cat /proc/krping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>1 listen: indicates the server is waiting<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>1-amso0 n1 n2 n3 ...: indicates the connection has been
made<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>If n = 0 for all n, there is probably an error<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>_____<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>rping<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>-----<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Server: rping -s -vV -C10 -S10 -a 0.0.0.0 -p 9999<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>Client: rping -c -vV -C10 -S10 -a 192.168.69.149 -p 9999<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>This should output:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'>ping data : rdma-ping (10x because of -C10)<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Courier><span style='font-size:10.0pt;
font-family:Courier'><o:p> </o:p></span></font></p>
</div>
</body>
</html>