[openib-general] [PATCH 0/4] SA path record caching

[Or Gerlitz]

Sean Hefty wrote:
Or >> Aren't we creating a monster here??? if this is SA replica which 
should
Or >> work for scale from day one, lets call it this way and see how to 
reach there

> The cache update window is configurable.  
...
> Based on information from SilverStorm, the cache should work well in practice.

I recall that SilverStorm mentioned they had a well working SA replica, 
but when i look on the naked math for 1k nodes and the hard to reach in 
real life uniform distribution of queries over time, i really can't see 
how to reach it unless you practically never update the cache and you 
have magically caused your SM/SA to survive the one and only session of 
those 1k get table queries (again 350 mad/sec, so many concurrent rmpp 
sessions etc etc).

>> + neither MVAPICH nor OpenMPI are using path query

> The national labs want all path records for their routing algorithms.  I

So your understanding they have an --all to all- 1k ranks IB app that 
needs to know for each node all the paths to other nodes? if the case is 
1k IB app where each rank connects to (say log2(1k)=) 10 ranks then each 
rank needs all the paths to those 10 other nodes.

 > I believe that the problems here were API issues that make connecting
 > difficult.  As a result, most applications just hard-coded everything.

I guess feedback from MPI people telling whether they have plans to use 
path query would help us to see where we actually stand.

Or.