[ofa-general] [PATCH] infiniband-diags/scripts: Add ibcheckroutes to scripts
Yevgeny Kliteynik
kliteyn at dev.mellanox.co.il
Mon Sep 14 15:25:12 PDT 2009
Hal Rosenstock wrote:
>
>
> On Mon, Sep 14, 2009 at 11:05 AM, Eli Dorfman (Voltaire)
> <dorfman.eli at gmail.com <mailto:dorfman.eli at gmail.com>> wrote:
>
> Hal Rosenstock wrote:
> >
> >
> > On Mon, Sep 14, 2009 at 10:32 AM, Eli Dorfman (Voltaire)
> > <dorfman.eli at gmail.com <mailto:dorfman.eli at gmail.com>
> <mailto:dorfman.eli at gmail.com <mailto:dorfman.eli at gmail.com>>> wrote:
> >
> > Hal Rosenstock wrote:
> > >
> > >
> > > On Sun, Sep 13, 2009 at 3:55 AM, Doron Shoham
> <dorons at voltaire.com <mailto:dorons at voltaire.com>
> > <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>>
> > > <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>
> <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>>>> wrote:
> > >
> > > Hal Rosenstock wrote:
> > > >
> > > >
> > > > On Thu, Sep 10, 2009 at 7:56 AM, Doron Shoham
> > <dorons at voltaire.com <mailto:dorons at voltaire.com>
> <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>>
> > > <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com> <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com>>>
> > > > <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com> <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com>>
> > <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>
> <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>>>>> wrote:
> > > >
> > > > ibcheckroutes validates route between all hosts
> in the
> > fabric.
> > > > This script finds all leaf switches (switches
> that are
> > > connected to
> > > > HCAs)
> > > >
> > >
> > > This script parses the output of ibnetdiscoer.
> > > It finds all leaf switches (from the topology file
> > > generated by ibnetdiscover).
> > > The it checks if a route exists between all leaf switches
> > > using ibtracert.
> > >
> > >
> > > Why leaf switches (and not CAs) ? How are they determined
> (from the
> > > ibnetdiscover output) ?
> >
> >
> >
> > How are the leaf switches determined (from core switches) in the
> > ibnetdiscover output ? Is it any switch which has an attached CA
> versus
> > any switch which has no attached CAs ?
> yes
>
> >
> >
> >
> > because there are much less combinations (routes) of leaf
> switches
> > than CAs.
> >
> >
> > So is the check is that there are routes between all the leaf
> switches ?
> yes
> >
> >
> > And since we assume that opensm routing builds lid matrix
> based on
> > switch connectivity than if two switches have route between each
> > other then all CAs that are connected to them will have route to
> > each other.
> >
> >
> > I can't parse this sentence. Also, this should have nothing to do
> with
> > OpenSM as it is SM independent AFAIT.
> since we check routes between leaf switches LIDs, for opensm this
> also assures that we have route between CAs that are attached to them.
>
>
> Not quite. It assures there should be a path but not necessarily a route
> as all the LFTs are not checked with the CA port LIDs.
One such example is ftree routing.
First it creates all the CA-2-CA routes, and only after this
it creates switch-2-switch routing. So in case of ftree routing,
having full connectivity between leaf switches doesn't imply
anything w.r.t. CA-2-CA connectivity.
-- Yevgeny
> -- Hal
>
>
>
> >
> >
> > In ibnetdiscover you can see to which switch (LID) each CA is
> connected.
> >
> >
> > Sure.
> >
> >
> >
> > >
> > >
> > >
> > > >
> > > > CAs or HCAs ?
> > > CAs
> > > >
> > > > What about switch port 0s ?
> > > It checks connectivity only between leaf switches (not all
> > switches).
> > > I assume that traffic is generated only between CAs and
> therefor
> > > connectivity between other switches (not leaf switches)
> does not
> > > important.
> > >
> > >
> > > It's important for a couple of reasons: first PMA access on
> > switches and
> > > secondly it's an IBA requirement although some OpenSM routing
> > protocols
> > > ignore this. IMO it should be an option (not the default)
> to add these
> > > LIDs in too to the ones checked.
> >
> > Ok, we can this option once this patch is applied.
> >
> >
> > I have some other specific comments on the patch.
> >
> >
> > Also it may be better to provide the switch LID(s) from which
> PM is
> > running to reduce number of tested routes.
> >
> >
> > This is in the vein of only checking leaf switch connectivity but
> is not
> > the IBA general requirement.
> >
> > -- Hal
> >
> >
> >
> > Eli
> >
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > > and runs ibtracert between them.
> > > > When using various routing algorithms (e.g. up-down),
> > > >
> > > >
> > > > With which routing algorithms has this been tried ?
> > > I assume that from complexity perspective, the routing
> algorithms
> > > calculate
> > > routes only between leaf switches and not between all CAs.
> > > Then it adds one hop for all CAs connected to the leaf
> switches.
> > >
> > >
> > > It depends on the routing algorithm (some violate this) but
> the basic
> > > IBA requirement is:
> > > *
> > >
> > > C14-62.1.4:
> > >
> > > *From every endport within the subnet, the SM *shall
> *provide at least
> > > one reversible path to every other endport.
> > >
> > > -- Hal
> > >
> > >
> > > I've tested it with up-down but it really doesn't
> matter which
> > > routing algorithm you are using.
> > > It just check the routes between leaf switches (and if
> the routing
> > > algorithm behave as above, it means that it checks all CAs
> > > connectivity).
> > >
> > > >
> > > > -- Hal
> > > >
> > > >
> > > > if fabric topology is not suitable there will be no
> > > > routes between some nodes.
> > > > It reports when the route exists between source and
> > > destination LIDs.
> > > >
> > > > Signed-off-by: Doron Shoham <dorons at voltaire.com
> <mailto:dorons at voltaire.com>
> > <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>>
> > > <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com> <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com>>>
> > > > <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com> <mailto:dorons at voltaire.com
> <mailto:dorons at voltaire.com>>
> > <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>
> <mailto:dorons at voltaire.com <mailto:dorons at voltaire.com>>>>>
> > > >
> > > >
> > > > <snip...>
> > >
> > >
> > >
> > >
> >
> ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > general mailing list
> > > general at lists.openfabrics.org
> <mailto:general at lists.openfabrics.org>
> <mailto:general at lists.openfabrics.org
> <mailto:general at lists.openfabrics.org>>
> > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> > >
> > > To unsubscribe, please visit
> > http://openib.org/mailman/listinfo/openib-general
> >
> >
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
More information about the general
mailing list