[ofa-general] Re: [RFC] [PATCH 0/3] 2.6.22 or 23 ib: add path record cache

[Michael S. Tsirkin]

> Quoting Sean Hefty <sean.hefty at intel.com>:
> Subject: RE: [ofa-general] Re: [RFC] [PATCH 0/3] 2.6.22 or 23 ib: add path record cache
> 
> >Has anyone thought about using replication rather than caching to
> >solve this problem? It seems to me it would be alot faster for some
> >single process in the network to fetch and keep a copy of the entire
> >SA route database, format it into a binary format and use RC RDMA to
> >transfer it to every node each time it changes.
> 
> I have given thought to using RC RDMA to distribute the data to all nodes,
> especially to eliminate the MAD protocol overhead.  There are a couple issues
> with this:
> 
> To work with existing SAs, we need to working within the defined SA interface
> (i.e. SA MADs), so something still needs to query for all path records.
> 
> The GetTable query requires an SGID, which means that whatever node collects the
> path records must first collect all the GIDs.  (And the most efficient way I've
> found to obtain a list of all GIDs is via a GetTable path record query...)  This
> also means that the node collecting the path records will generate 1 query per
> GID.  This has the same impact on the SA as each node issuing their own query.
> And the impact on the subnet is higher, since we still need to distribute that
> data to the end nodes.

We could solve this by implementing a process running on the same node as the SA.
And it's probably not too hard to add a way for opensm to spit out
the table into an external file when it gets a signal or something.


-- 
MST