[openib-general] SA cache design

[Eitan Zahavi]

> 
> 
> On a related note, why does every instance of the application need to
query for
> every other instance?  To establish all-to-all communication, couldn't
instance
> X only initiate connections to instances > X?  (I.e. 1 connects to 2
and 3, 2
> connects to 3.)
[EZ] MPI opens a connection from each node to every other node. Actually
even from every CPU to every other CPU. So this is why we have N^2
connections.
> 
> > Only a very small subset of queries is used:
> > * PathRecord by SRC-GUID,DST-GUID
> > * PortInfo by capability mask
> 
> I did look at the code to see what queries were actually being used
today.  And
> yes, we can implement for only those cases.  I wanted to allow the
flexibility
> to support other queries efficiently.
[EZ] The scalability issues we see today are what I most worry about.
> 
> - Sean