<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 10 (filtered)">
<title>RE: [Openib-windows] [RFC] changing kernel complib to be static library</title>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
p
{margin-right:0in;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
span.EmailStyle18
{font-family:Arial;
color:navy;}
@page Section1
{size:595.3pt 841.9pt;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=EN-US link=blue vlink=blue>
<div class=Section1 dir=RTL>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>I disagree with most of
the drawbacks that you have mentioned. Please note that there is no code
duplication, but rather using the same compiled code a few times.</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>As a result this means for
(3) that if there is a change in the initialization/termination sequence, there
is nothing that should be changed in other ULPs that we don't have to change
today.</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>For 3'rd partied users:</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>ULPs that we do have their
code, we can simply recompile them as we do with a DLL. As for 3'rd parties ULPs
that we don't have their code: Using a DLL they just crush as the dll has just
changed, and they are not ready for this. If they are using a static lib, they
can still work (nothing has changed for them).</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>The real drawback is the
size that is increasing, but I don't think that this is a real issue.</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>On the other hand most of
the problems can be avoided (still using a shared DLL) by 1) always installing
both shared free/chk dlls. In this case everyone can use what ever he wants. 2)
Changing complib not to have the structures that their size changes depending
if this is a chk or free build.</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>In any case, the current situation
should change.</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>Thanks</span></font></p>
<p class=MsoNormal dir=LTR><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'>Tzachi</span></font></p>
<p class=MsoNormal dir=LTR style='margin-left:.5in'><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma'>-----Original Message-----<br>
<b><span style='font-weight:bold'>From:</span></b> Leonid Keller [mailto:leonid@mellanox.co.il]
<br>
<b><span style='font-weight:bold'>Sent:</span></b> </span></font><font size=2 face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'>Thursday, July
14, 2005</span></font><font size=2 face=Tahoma><span style='font-size:10.0pt;
font-family:Tahoma'> </span></font><font size=2 face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma'>2:55 AM</span></font><font size=2
face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma'><br>
<b><span style='font-weight:bold'>To:</span></b> 'Fab Tillier'; Leonid Keller;
openib-windows@openib.org<br>
<b><span style='font-weight:bold'>Subject:</span></b> RE: [Openib-windows]
[RFC] changing kernel complib to be static l ibrary</span></font></p>
<p class=MsoNormal dir=LTR style='margin-left:.5in'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'> </span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>Checking the build type is just a temporary
workaround, that will be removed after adding support to debug/release mix
(which, i agree, is very worthful).</span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>COMPLIB as a static library has more drawbacks: </span></font><br>
<font size=2><span style='font-size:
10.0pt'>1. Unsynchronized common mechanisms</span></font> <br>
<font size=2><span style='font-size:
10.0pt'> Imagine, that you want to synchronize several components
by taking some well-known sync object.</span></font> <br>
<font size=2><span style='font-size:
10.0pt'> You will possibly implement the support to this object in
COMPLIB.</span></font> <br>
<font size=2><span style='font-size:
10.0pt'> If every component uses its own copy of complib, i.e - its
own implementation of this mechanism - you can get into bad troubles.</span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> </span></font><font
size=2><span style='font-size:10.0pt'>2. 3rd party users</span></font> <br>
<font size=2><span style='font-size:
10.0pt'> What if some other company want to develop a ULP, using
our COMPLIB. </span></font><br>
<font size=2><span style='font-size:
10.0pt'> How do you syncronize you bugfixing/developing of COMPLIB
with him ?</span></font> <br>
<font size=2><span style='font-size:
10.0pt'>3. Multiplied code</span></font> <br>
<font size=2><span style='font-size:
10.0pt'> Imagine, that you change for some reason COMPLIB's
initialization/termination sequence.</span></font> <br>
<font size=2><span style='font-size:
10.0pt'> Now one needs to fix the appropiate code in ALL the
components, using it, which is errorprone. </span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>A DLL version check fully prevents ALL weird
situations, caused - in the first place - by code duplication. </span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>I agree with turning a single-used DLL into a static
library, but i still believe, that a commonly used component would rather be a
DLL (with an *obligatory* version checking mechanism).</span></font></p>
<p class=MsoNormal dir=LTR style='margin-left:.5in'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'> </span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>-----Original Message-----</span></font> <br>
<font size=2><span style='font-size:10.0pt'>From: Fab Tillier [<a
href="mailto:ftillier@silverstorm.com">mailto:ftillier@silverstorm.com</a>]</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>Sent: Thursday, July 14, 2005 4:30
AM</span></font> <br>
<font size=2><span style='font-size:10.0pt'>To: 'Leonid Keller';
openib-windows@openib.org</span></font> <br>
<font size=2><span style='font-size:10.0pt'>Subject: RE: [Openib-windows] [RFC]
changing kernel complib to be static</span></font> <br>
<font size=2><span style='font-size:10.0pt'>library</span></font> </p>
<p class=MsoNormal dir=LTR style='margin-left:.5in'><font size=3
face="Times New Roman"><span style='font-size:12.0pt'> </span></font></p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>> From: Leonid Keller [<a
href="mailto:leonid@mellanox.co.il">mailto:leonid@mellanox.co.il</a>]</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>> Sent: Wednesday, July 13, 2005
10:06 AM</span></font> <br>
<font size=2><span style='font-size:10.0pt'>> </span></font><br>
<font size=2><span style='font-size:10.0pt'>> i don't like the idea.</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>> Windows' kernel consists of
DLLs and they change it with service packs</span></font> <br>
<font size=2><span style='font-size:10.0pt'>> anyway ...</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>Yes, and they require a reboot anytime that happens,
which everyone complains</span></font> <br>
<font size=2><span style='font-size:10.0pt'>about. However, this is for
core kernel components. We're a loadable driver</span></font> <br>
<font size=2><span style='font-size:10.0pt'>and should do everything we can to
avoid a reboot.</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>> And what about IBAL.sys - it is also used now by
several components ?</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>IBAL.sys is only used by ibbus.sys and by
thca.sys. Every other driver uses bus</span></font> <br>
<font size=2><span style='font-size:10.0pt'>interface mechanisms rather than
linking directly. THCA should be changed to</span></font> <br>
<font size=2><span style='font-size:10.0pt'>use the interface too, but hasn't
been. I believe the only uses of ibal.sys in</span></font> <br>
<font size=2><span style='font-size:10.0pt'>thca.sys are for generating nice
error messages (ib_get_err_str), but I need to</span></font> <br>
<font size=2><span style='font-size:10.0pt'>check. On my todo list is
removing the DLL dependency between thca.sys and</span></font> <br>
<font size=2><span style='font-size:10.0pt'>ibal.sys, and then making ibal.sys
a static library linked into ibbus.sys (and</span></font> <br>
<font size=2><span style='font-size:10.0pt'>no where else).</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>> Using a static library can bring us to kind of
"DLL mess" problem, when each</span></font> <br>
<font size=2><span style='font-size:10.0pt'>> driver works with its own
version of COMPLIB and a bug, fixed in one version,</span></font> <br>
<font size=2><span style='font-size:10.0pt'>> continues to be present in
others.</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>True, however such bugs should be infrequent.
Further, bugs in inline functions</span></font> <br>
<font size=2><span style='font-size:10.0pt'>will not be resolved without
rebuilding all drivers that use those functions.</span></font> <br>
<font size=2><span style='font-size:10.0pt'>The goal isn't to allow mixing
versions of the drivers, but rather allowing</span></font> <br>
<font size=2><span style='font-size:10.0pt'>mixing debug and release
drivers. The OS currently allows this - you can</span></font> <br>
<font size=2><span style='font-size:10.0pt'>replace any driver in the system
with its debug counterpart and end up with a</span></font> <br>
<font size=2><span style='font-size:10.0pt'>fully functional system. That
is not the case with our IB drivers, and IMO this</span></font> <br>
<font size=2><span style='font-size:10.0pt'>is a pretty serious flaw. The
alternative is to use macro-magic and provide</span></font> <br>
<font size=2><span style='font-size:10.0pt'>support for both debug and release
clients at the same time regardless of the</span></font> <br>
<font size=2><span style='font-size:10.0pt'>build type of complib.sys, however
this is rather cumbersome.</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>Take quick list for example. You would have to
create a __qlist_item_fre_t and</span></font> <br>
<font size=2><span style='font-size:10.0pt'>__qlist_item_chk_t structures, one
for what a list item looks like on a free</span></font> <br>
<font size=2><span style='font-size:10.0pt'>build, the other for a checked
build. Then cl_list_item_t would be #defined</span></font> <br>
<font size=2><span style='font-size:10.0pt'>like this:</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>#ifdef _DEBUG_</span></font> <br>
<font size=2><span style='font-size:10.0pt'>#define cl_list_item_t
__qlist_item_chk_t</span></font> <br>
<font size=2><span style='font-size:10.0pt'>#else</span></font> <br>
<font size=2><span style='font-size:10.0pt'>#define cl_list_item_t
__qlist_item_fre_t</span></font> <br>
<font size=2><span style='font-size:10.0pt'>#endif</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>Same goes for all functions that operate on any
structure that changes between</span></font> <br>
<font size=2><span style='font-size:10.0pt'>debug and release builds (which
many do).</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>In any case, we need to make it so that someone
doesn't hose their system by</span></font> <br>
<font size=2><span style='font-size:10.0pt'>forgetting to copy a driver.</span></font>
</p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>> May be we can use another solutions, say:</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>>
- add versioning to COMPLIB , so a new HCA driver will fail loading</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>> upon unappropriate COMPLIB
version;</span></font> <br>
<font size=2><span style='font-size:10.0pt'>>
- add checking of build_types of the components (also on startup);</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>>
- (the worst case) to copy all the files to system\drivers upon driver</span></font>
<br>
<font size=2><span style='font-size:10.0pt'>> update and require reboot! :(</span></font>
</p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>I don't like the idea of performing runtime checks for
linkage issues especially</span></font> <br>
<font size=2><span style='font-size:10.0pt'>since the check would have to check
both version and build type. It wouldn't be</span></font> <br>
<font size=2><span style='font-size:10.0pt'>possible to mix debug and release
drivers, which is something that can be done</span></font> <br>
<font size=2><span style='font-size:10.0pt'>today, and can be very valuable by
letting you test a single driver without</span></font> <br>
<font size=2><span style='font-size:10.0pt'>incurring the timing change penalty
of running the full debug stack.</span></font> </p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>Requiring a reboot does not provide a good user
experience, so we should avoid</span></font> <br>
<font size=2><span style='font-size:10.0pt'>it if at all possible.</span></font>
</p>
<p dir=LTR style='margin-left:.5in'><font size=2 face="Times New Roman"><span
style='font-size:10.0pt'>- Fab</span></font> </p>
</div>
</body>
</html>