[ofa-general] Re: [Query] ib add path record cache

[Hal Rosenstock]

On Thu, 2007-05-17 at 08:28, Devesh Sharma wrote:
> On 17 May 2007 06:42:16 -0400, Hal Rosenstock <halr at voltaire.com> wrote:
> > On Thu, 2007-05-17 at 01:21, Devesh Sharma wrote:
> > > On 5/17/07, Sean Hefty <mshefty at ichips.intel.com> wrote:
> > > > > But initially this will generate a packet for each path, while sys
> > > > > admin knows that path is there and he can hard-code the entries for
> > > > > it. Other thing is that why Admin will care about creating such record
> > > > > while SA is itself taking care, right?
> > > >
> > > > In your original message you asked about adding 'dummy entries' to the
> > > > cache.  I agree that pre-loading the cache can be useful.  What I still
> > > > am not understanding is the reasoning for adding 'dummy entries'.  By
> > > > 'dummy entries', I've been assuming that these are invalid path records,
> > > > but maybe that's not what you meant.
> > > Ok if "dummy entries" word as such has created confusion then I am
> > > sorry for that, But with that I mean that, those are valid path
> > > records which Administrator knows in advance and while loading the
> > > module,
> >
> > How does the admin know they are valid ?
> Depending on the initial application runs, some trusted PRs can be generated.

What do initial application runs have to do with this ?

> >Are they somehow preconfigured at the SM ?
> I am not sure about SM has any such provision?

Not that I'm aware of.

> Also not sure about the
> role of SM in path resolving. I mean once node has initiated SA query,
> whether SM has some database to reply SA or On the fly destination
> node is contacted to get asked path recored?

SMs can either calculate the SA PRs on the fly based on the routing
algorithm in use and some other things or put them in a local database.
This is up to that SM.

Destination node is not contacted in the SA PR query process.

> >Doesn't each SM have its own policy for generating valid PRs ?
> Ultimately path record is in Path_Record object format, and SA cache
> is going to store in a fixed manner, How generation policy matters?

What if the local policy loaded does not agree with what the SM would
generate for a particular PR ? One then gets a local error which will
need to be tracked down. Not so easy IMO.

> CMIIW. Also I am assuming a homogeneous cluster where certain
> parameters can be assumed to be same always.

and always in agreement with what the SM would return ? For example,
what happens when a link goes down and the end node is no longer
reachable ?

> >are these from a live SM and just loaded "out of band" to
> bypass/preclude the SA PR >mechanism ?
> may be

Even if they are, there is still the changes in the subnet issue.

-- Hal

> > -- Hal
> >
> > >  Admin is loading this info in the cache with user command.
> > > >
> > > > > Another point I want to know is,
> > > > > When local_sa_cache module will be inserted? After SM comes up or
> > > > > Before SM comes up?
> > > >
> > > > It can occur either way.  There is no restriction.  The cache responds
> > > > to port up and GID in/out of service events to update itself.
> > > Do you mean cache module will start building cache only after Port is UP?
> > > >
> > > > > If Its inserted before SM is coming up (I am assuming SM is running on
> > > > > some node not on switch) then First Forced schedule_update() is
> > > > > waisted, and for the first application presence of cache is
> > > > > meaningless. Why not to keep cache effective right from the start?
> > > >
> > > > Pre-loading the cache with path records doesn't guarantee that those
> > > > paths are usable.  If the SM has not come up, then the path records will
> > > > be unusable until the SM configures the subnet, plus there's no
> > > > guarantee that the remote endpoints specified by the paths are running.
> > > You mean there is no guarantee that even if SM is UP and we have some
> > > hard coded entries of path record corresponding to some node X, we are
> > > not sure that node X has actually come up or not?  In that case
> > > actually that path resolving should fail if node has not come up, but
> > > with the hard coding still path will be resolved?
> > > >
> > > > The main benefit I see to pre-loading the cache is to avoid SA storms
> > > > when booting a large cluster.
> > > that's true. Also cache will get valid entries only if network is
> > > configured by SM otherwise every node SA will, possibly, drop SA
> > > packets.
> > > >
> > > > - Sean
> > > >
> > > _______________________________________________
> > > general mailing list
> > > general at lists.openfabrics.org
> > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> > >
> > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
> >
> >