[ofa-general] Re: [RFC] [PATCH 0/3] 2.6.22 or 23 ib: add path record cache

[Hal Rosenstock]

On Mon, 2007-04-23 at 18:05, Sean Hefty wrote:
> >Has anyone thought about using replication rather than caching to
> >solve this problem? It seems to me it would be alot faster for some
> >single process in the network to fetch and keep a copy of the entire
> >SA route database, format it into a binary format and use RC RDMA to
> >transfer it to every node each time it changes.
> 
> I have given thought to using RC RDMA to distribute the data to all nodes,
> especially to eliminate the MAD protocol overhead.  There are a couple issues
> with this:
> 
> To work with existing SAs, we need to working within the defined SA interface
> (i.e. SA MADs), so something still needs to query for all path records.
> 
> The GetTable query requires an SGID, which means that whatever node collects the
> path records must first collect all the GIDs.  (And the most efficient way I've
> found to obtain a list of all GIDs is via a GetTable path record query...)  This
> also means that the node collecting the path records will generate 1 query per
> GID.  This has the same impact on the SA as each node issuing their own query.
> And the impact on the subnet is higher, since we still need to distribute that
> data to the end nodes.
> 
> In short, until we can standardize on some new SA interface, or we have a
> distributed SA, I don't see where we can do much better than caching GetTable
> responses at the end nodes.

Or perhaps an additional standard for replication. I think distributed
SA is a separate related issue.

-- Hal

> - Sean
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general