<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<TITLE>RE: [Openib-windows] [RFC] changing kernel complib to be static library</TITLE>
<META content="MSHTML 5.00.2314.1000" name=GENERATOR>
<STYLE>@font-face {
font-family: Tahoma;
}
P.MsoNormal {
FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt
}
LI.MsoNormal {
FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt
}
DIV.MsoNormal {
FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt; MARGIN: 0in 0in 0pt
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: blue; TEXT-DECORATION: underline
}
P {
FONT-FAMILY: "Times New Roman"; FONT-SIZE: 12pt; MARGIN-LEFT: 0in; MARGIN-RIGHT: 0in
}
SPAN.EmailStyle18 {
COLOR: navy; FONT-FAMILY: Arial
}
DIV.Section1 {
page: Section1
}
</STYLE>
</HEAD>
<BODY lang=EN-US link=blue vLink=blue>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=515403817-14072005>see
below</SPAN></FONT></DIV>
<BLOCKQUOTE dir=rtl style="MARGIN-LEFT: 0px">
<DIV align=left class=OutlookMessageHeader dir=ltr><FONT face=Tahoma
size=2>-----Original Message-----<BR><B>From:</B> Tzachi Dar <BR><B>Sent:</B>
Thursday, July 14, 2005 5:15 PM<BR><B>To:</B> Leonid Keller; 'Fab Tillier';
openib-windows@openib.org<BR><B>Subject:</B> RE: [Openib-windows] [RFC]
changing kernel complib to be static l ibrary<BR><BR></DIV></FONT>
<DIV class=Section1 dir=rtl>
<P class=MsoNormal dir=ltr><FONT color=navy><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">I disagree with most
of the drawbacks that you have mentioned. Please note that there is no code
duplication, but rather using the same compiled code a few times.<BR><FONT
color=#0000ff><SPAN class=515403817-14072005>[Leonid Keller] Yes, it's
obvious. I was talking about a siuation, that for some reason only a part of
the drivers get replaced and different communicating components are working
each with its own version of COMPLIB</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt"> </SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">As a result this
means for (3) that if there is a change in the initialization/termination
sequence, there is nothing that should be changed in other ULPs that we don't
have to change today.<BR><FONT color=#0000ff><SPAN
class=515403817-14072005>[Leonid Keller] Fab mentioned, that changing DLL
to static library requires adding initialization/termination code in every
componet, using COMPLIB, so ...</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt"> </SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">For 3'rd partied
users:</SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">ULPs that we do have
their code, we can simply recompile them as we do with a DLL. As for 3'rd
parties ULPs that we don't have their code: Using a DLL they just crush as the
dll has just changed, and they are not ready for this. If they are using a
static lib, they can still work (nothing has changed for them).<BR><FONT
color=#0000ff><SPAN class=515403817-14072005>[Leonid Keller] NO !!! They
must perform version checking and fail their startup
! </SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt"> </SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">The real drawback is
the size that is increasing, but I don't think that this is a real
issue.</SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">On the other hand
most of the problems can be avoided (still using a shared DLL) by 1) always
installing both shared free/chk dlls. In this case everyone can use what ever
he wants. 2) Changing complib not to have the structures that their size
changes depending if this is a chk or free build.<BR><FONT color=#0000ff><SPAN
class=515403817-14072005>[Leonid Keller] Fab told about the
situation, that release and debug versions were accidentally
mixed.</SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt"><FONT
color=#0000ff><SPAN class=515403817-14072005>But here, as i take, all agree,
that we need to perform the necessary changes to enable that
mix. </SPAN></FONT></SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt"> </SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">In any case, the
current situation should change.</SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt"> </SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">Thanks</SPAN></FONT></P>
<P class=MsoNormal dir=ltr><FONT color=navy face=Arial size=2><SPAN
style="COLOR: navy; FONT-FAMILY: Arial; FONT-SIZE: 10pt">Tzachi</SPAN></FONT></P>
<P class=MsoNormal dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face=Tahoma
size=2><SPAN style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt">-----Original
Message-----<BR><B><SPAN style="FONT-WEIGHT: bold">From:</SPAN></B> Leonid
Keller [mailto:leonid@mellanox.co.il] <BR><B><SPAN
style="FONT-WEIGHT: bold">Sent:</SPAN></B> </SPAN></FONT><FONT face=Tahoma
size=2><SPAN style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt">Thursday, July 14,
2005</SPAN></FONT><FONT face=Tahoma size=2><SPAN
style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt"> </SPAN></FONT><FONT face=Tahoma
size=2><SPAN style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt">2:55
AM</SPAN></FONT><FONT face=Tahoma size=2><SPAN
style="FONT-FAMILY: Tahoma; FONT-SIZE: 10pt"><BR><B><SPAN
style="FONT-WEIGHT: bold">To:</SPAN></B> 'Fab Tillier'; Leonid Keller;
openib-windows@openib.org<BR><B><SPAN
style="FONT-WEIGHT: bold">Subject:</SPAN></B> RE: [Openib-windows] [RFC]
changing kernel complib to be static l ibrary</SPAN></FONT></P>
<P class=MsoNormal dir=ltr style="MARGIN-LEFT: 0.5in"><FONT
face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> </SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">Checking the build type is just a
temporary workaround, that will be removed after adding support to
debug/release mix (which, i agree, is very worthful).</SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">COMPLIB as a static library has more
drawbacks: </SPAN></FONT><BR> <FONT
size=2><SPAN style="FONT-SIZE: 10pt">1. Unsynchronized common
mechanisms</SPAN></FONT> <BR> <FONT
size=2><SPAN style="FONT-SIZE: 10pt"> Imagine, that you want to
synchronize several components by taking some well-known sync
object.</SPAN></FONT> <BR> <FONT
size=2><SPAN style="FONT-SIZE: 10pt"> You will possibly implement
the support to this object in COMPLIB.</SPAN></FONT>
<BR> <FONT size=2><SPAN
style="FONT-SIZE: 10pt"> If every component uses its own copy of
complib, i.e - its own implementation of this mechanism - you can get into bad
troubles.</SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=3><SPAN
style="FONT-SIZE: 12pt">
</SPAN></FONT><FONT size=2><SPAN style="FONT-SIZE: 10pt">2. 3rd party
users</SPAN></FONT> <BR> <FONT
size=2><SPAN style="FONT-SIZE: 10pt"> What if some other company
want to develop a ULP, using our COMPLIB.
</SPAN></FONT><BR> <FONT
size=2><SPAN style="FONT-SIZE: 10pt"> How do you syncronize you
bugfixing/developing of COMPLIB with him ?</SPAN></FONT>
<BR> <FONT size=2><SPAN
style="FONT-SIZE: 10pt">3. Multiplied code</SPAN></FONT>
<BR> <FONT size=2><SPAN
style="FONT-SIZE: 10pt"> Imagine, that you change for some reason
COMPLIB's initialization/termination sequence.</SPAN></FONT>
<BR> <FONT size=2><SPAN
style="FONT-SIZE: 10pt"> Now one needs to fix the appropiate code
in ALL the components, using it, which is errorprone. </SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">A DLL version check fully prevents ALL
weird situations, caused - in the first place - by code duplication.
</SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">I agree with turning a single-used DLL
into a static library, but i still believe, that a commonly used component
would rather be a DLL (with an *obligatory* version checking
mechanism).</SPAN></FONT></P>
<P class=MsoNormal dir=ltr style="MARGIN-LEFT: 0.5in"><FONT
face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> </SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">-----Original Message-----</SPAN></FONT>
<BR><FONT size=2><SPAN style="FONT-SIZE: 10pt">From: Fab Tillier [<A
href="mailto:ftillier@silverstorm.com">mailto:ftillier@silverstorm.com</A>]</SPAN></FONT>
<BR><FONT size=2><SPAN style="FONT-SIZE: 10pt">Sent: Thursday, July 14, 2005
4:30 AM</SPAN></FONT> <BR><FONT size=2><SPAN style="FONT-SIZE: 10pt">To:
'Leonid Keller'; openib-windows@openib.org</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">Subject: RE: [Openib-windows] [RFC]
changing kernel complib to be static</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">library</SPAN></FONT> </P>
<P class=MsoNormal dir=ltr style="MARGIN-LEFT: 0.5in"><FONT
face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> </SPAN></FONT></P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">> From: Leonid Keller [<A
href="mailto:leonid@mellanox.co.il">mailto:leonid@mellanox.co.il</A>]</SPAN></FONT>
<BR><FONT size=2><SPAN style="FONT-SIZE: 10pt">> Sent: Wednesday, July 13,
2005 10:06 AM</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> </SPAN></FONT><BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> i don't like the idea.</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">> Windows' kernel consists of DLLs and
they change it with service packs</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> anyway ...</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">Yes, and they require a reboot anytime
that happens, which everyone complains</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">about. However, this is for core kernel
components. We're a loadable driver</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">and should do everything we can to avoid a
reboot.</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">> And what about IBAL.sys - it is also
used now by several components ?</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">IBAL.sys is only used by ibbus.sys and by
thca.sys. Every other driver uses bus</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">interface mechanisms rather than linking
directly. THCA should be changed to</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">use the interface too, but hasn't been. I
believe the only uses of ibal.sys in</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">thca.sys are for generating nice error messages
(ib_get_err_str), but I need to</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">check. On my todo list is removing the DLL
dependency between thca.sys and</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">ibal.sys, and then making ibal.sys a static library
linked into ibbus.sys (and</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">no where else).</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">> Using a static library can bring us
to kind of "DLL mess" problem, when each</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> driver works with its own version of COMPLIB and
a bug, fixed in one version,</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> continues to be present in others.</SPAN></FONT>
</P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">True, however such bugs should be
infrequent. Further, bugs in inline functions</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">will not be resolved without rebuilding
all drivers that use those functions.</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">The goal isn't to allow mixing versions of the
drivers, but rather allowing</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">mixing debug and release drivers. The OS
currently allows this - you can</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">replace any driver in the system with its debug
counterpart and end up with a</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">fully functional system. That is not the case
with our IB drivers, and IMO this</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">is a pretty serious flaw. The alternative is to use
macro-magic and provide</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">support for both debug and release clients at the same
time regardless of the</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">build type of complib.sys, however this is rather
cumbersome.</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">Take quick list for example. You
would have to create a __qlist_item_fre_t and</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">__qlist_item_chk_t structures, one for
what a list item looks like on a free</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">build, the other for a checked build. Then
cl_list_item_t would be #defined</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">like this:</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">#ifdef _DEBUG_</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">#define cl_list_item_t
__qlist_item_chk_t</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">#else</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">#define cl_list_item_t
__qlist_item_fre_t</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">#endif</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">Same goes for all functions that operate
on any structure that changes between</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">debug and release builds (which many
do).</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">In any case, we need to make it so that
someone doesn't hose their system by</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">forgetting to copy a driver.</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">> May be we can use another solutions,
say:</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> -
add versioning to COMPLIB , so a new HCA driver will fail
loading</SPAN></FONT> <BR><FONT size=2><SPAN style="FONT-SIZE: 10pt">> upon
unappropriate COMPLIB version;</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> -
add checking of build_types of the components (also on startup);</SPAN></FONT>
<BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">> -
(the worst case) to copy all the files to system\drivers upon
driver</SPAN></FONT> <BR><FONT size=2><SPAN style="FONT-SIZE: 10pt">>
update and require reboot! :(</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">I don't like the idea of performing
runtime checks for linkage issues especially</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">since the check would have to check both
version and build type. It wouldn't be</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">possible to mix debug and release
drivers, which is something that can be done</SPAN></FONT> <BR><FONT
size=2><SPAN style="FONT-SIZE: 10pt">today, and can be very valuable by
letting you test a single driver without</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">incurring the timing change penalty of running the
full debug stack.</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">Requiring a reboot does not provide a
good user experience, so we should avoid</SPAN></FONT> <BR><FONT size=2><SPAN
style="FONT-SIZE: 10pt">it if at all possible.</SPAN></FONT> </P>
<P dir=ltr style="MARGIN-LEFT: 0.5in"><FONT face="Times New Roman"
size=2><SPAN style="FONT-SIZE: 10pt">- Fab</SPAN></FONT>
</P></DIV></BLOCKQUOTE></BODY></HTML>