<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2654.45">
<TITLE>RE: [Openib-windows] Re: Complib changes to support OpenSM in OpenIB</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>Hi fab, </FONT>
</P>
<P><FONT SIZE=2>I read the answer that you have sent me and I still disagree. </FONT>
<BR><FONT SIZE=2>Locking of objects and maintaining their life cycle is a very complicated task no matter what approach is used. I hope that we can agree at least on this.</FONT></P>
<P><FONT SIZE=2>As a result there are also many "valid" approaches on how to solve these problems. Deciding which approach is better has to take the following considerations: 1) simplicity of the code. 2) Performance of the code. 3) Time that is available for developing the code. 4) Existing code already. 5) Does the code has to be portable or not (and to where), and probably many other issues. Please also note, that when trying to decide on one of this issues, it is very hard to tell what are the "sub-considerations" that should be argued for example let's take performance: Are we measuring performance on a single processor or on a multiprocessor system? Are we measuring the performance when the system is loaded or not. Are we measuring the worst time complicity or the average complexity?</FONT></P>
<P><FONT SIZE=2>So far, many questions and not a single answer.</FONT>
<BR><FONT SIZE=2>I have wrote all this questions just to show that since the question is very complicated, it is hard to believe that it will have a simple answer. (I guess that this is the nice thing about computer since: although this "science" exists for more than 50 years, there is not one implementation of an array that is agreed on everyone).</FONT></P>
<P><FONT SIZE=2>The answer that you gave me bellow, if I understand it correctly, tells that it is possible to do better than always taking the lock (at some situations). Although I agree that it is some times true, I would like to suggest three other approaches to solving a programming problem in which this is not the case. What I mean by that is that using these approaches, one can write a "good" (what is really good, was not yet defined) program. Under this approaches one need a "multi-thread safe" container.</FONT></P>
<P><FONT SIZE=2>These approaches are:</FONT>
<BR><FONT SIZE=2>1) Use reference counting to mange the life time of the container, and use it from many threads. Personally I believe that when the project is big enough, this is the only thing that works, but this is just my opinion.</FONT></P>
<P><FONT SIZE=2>2) Use reference counting to manage the life time of more than one container and use them all.</FONT>
<BR><FONT SIZE=2>3) Have the container(s) accessed by more than one thread simultaneously and destroy it only when that thread is dead.</FONT>
</P>
<P><FONT SIZE=2>Again, I'm not saying that any of this approaches are "always better", but I do say that they might be used. As a result, I suggest that we will add the locked-pool to the complib implementation.</FONT></P>
<P><FONT SIZE=2>Thanks</FONT>
<BR><FONT SIZE=2>Tzachi</FONT>
</P>
<P><FONT SIZE=2>One other small point that I want to mention is this: weather to use inline or not inline is an implementation detail - although we might have to close this detail one day, I'm not sure that this is the time. </FONT></P>
<P><FONT SIZE=2>>-----Original Message-----</FONT>
<BR><FONT SIZE=2>>From: Fab Tillier [<A HREF="mailto:ftillier@silverstorm.com">mailto:ftillier@silverstorm.com</A>]</FONT>
<BR><FONT SIZE=2>>Sent: Friday, August 26, 2005 8:56 PM</FONT>
<BR><FONT SIZE=2>>To: 'Tzachi Dar'; Eitan Zahavi</FONT>
<BR><FONT SIZE=2>>Cc: Aviram Gutman; openib-windows@openib.org</FONT>
<BR><FONT SIZE=2>>Subject: RE: [Openib-windows] Re: Complib changes to support OpenSM in</FONT>
<BR><FONT SIZE=2>>OpenIB</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>> From: Tzachi Dar [<A HREF="mailto:tzachid@mellanox.co.il">mailto:tzachid@mellanox.co.il</A>]</FONT>
<BR><FONT SIZE=2>>> Sent: Friday, August 26, 2005 6:37 AM</FONT>
<BR><FONT SIZE=2>>></FONT>
<BR><FONT SIZE=2>>> Can you please explain in more detail what is wrong with the</FONT>
<BR><FONT SIZE=2>>cl_qlockpool.c ?</FONT>
<BR><FONT SIZE=2>>> It seems to me like a very common programming way that there is a</FONT>
<BR><FONT SIZE=2>>container</FONT>
<BR><FONT SIZE=2>>> that is single threaded, and in order to make it "multi-threaded save"</FONT>
<BR><FONT SIZE=2>>one</FONT>
<BR><FONT SIZE=2>>> adds a lock around it.</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>I have generally found that locking is required at a higher level than just</FONT>
<BR><FONT SIZE=2>>the</FONT>
<BR><FONT SIZE=2>>pool or list operations. There must be extra logic to prevent the pool</FONT>
<BR><FONT SIZE=2>>from</FONT>
<BR><FONT SIZE=2>>being destroyed while items are out, and locking at such a fine level</FONT>
<BR><FONT SIZE=2>>doesn't</FONT>
<BR><FONT SIZE=2>>help at all.</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>Even looking in the OSM code base, the few cases that I tried to figure out</FONT>
<BR><FONT SIZE=2>>held</FONT>
<BR><FONT SIZE=2>>other locks while accessing the pools. If there are other locks held, then</FONT>
<BR><FONT SIZE=2>>locking at the pool level is a waste of resources as the lock will never</FONT>
<BR><FONT SIZE=2>>see</FONT>
<BR><FONT SIZE=2>>contention. Having the lock hidden in the pool operations makes it much</FONT>
<BR><FONT SIZE=2>>less</FONT>
<BR><FONT SIZE=2>>obvious when the lock is unnecessary. If you see explicit locking, it's</FONT>
<BR><FONT SIZE=2>>much</FONT>
<BR><FONT SIZE=2>>easier to find out if the lock is needed or not, and if not, eliminated.</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>Lastly, if an operation does something like:</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>get item from pool</FONT>
<BR><FONT SIZE=2>>try something</FONT>
<BR><FONT SIZE=2>>if failed, put item in pool</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>If "try something" is a relatively quick operation, you'll get better</FONT>
<BR><FONT SIZE=2>>performance taking the lock up front and releasing it after the failure</FONT>
<BR><FONT SIZE=2>>test.</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>Also, as I mentioned before, having a function call just to mask a couple</FONT>
<BR><FONT SIZE=2>>function calls doesn't seem efficient. The qlockpool functions should be</FONT>
<BR><FONT SIZE=2>>inline.</FONT>
<BR><FONT SIZE=2>></FONT>
<BR><FONT SIZE=2>>- Fab</FONT>
</P>
</BODY>
</HTML>