[openib-general] SA cache design

[Sean Hefty]

Brian Long wrote:
> How much overhead is going to be incurred by using a standard RDBMS
> instead of not caching anything?  I'm not completely familiar with the
> IB configurations that would benefit from the proposed SA cache, but it
> seems to me, adding a RDBMS to anything as fast as IB would actually
> slow things down considerably.  Can an RDBMS + SA cache actually be
> faster than no cache at all?

I'm not sure what the speed-up of any cache will be.  The SA maintains a 
database of various related records - node records, path records, service 
records, etc. and responds to queries.  This need doesn't go away.  The SA 
itself is perfect candidate to be implemented using a DBMS.  (And if one had 
been implemented over a DBMS, I'm not even sure that we'd be talking about 
scalability issues for only a few thousand nodes.  Is the perceived lack of 
scalability of the SA a result of the architecture or the existing implementations?)

My belief is that a DBMS will outperform anything that I could write to store 
and retrieve these records.  Consider that a 4000 node cluster will have about 
8,000,000 path records.  Local caches can reduce this considerably (to about 
4000), and if we greatly restrict the type of queries that are supported, then 
we can manage the retrieval of those records ourselves.

I do not want end-users to have to administer a database.  However, if the user 
only needs to install a library, then this approach seems worth pursuing.

- Sean