[ofa-general] Re: OpenSM and fat tree

[Chris Worley]

On Thu, May 15, 2008 at 11:12 AM, Hal Rosenstock <hrosenstock at xsigo.com> wrote:
> On Thu, 2008-05-15 at 10:26 -0600, Chris Worley wrote:
>> On Thu, May 15, 2008 at 10:10 AM, Hal Rosenstock <hrosenstock at xsigo.com> wrote:
>> > Chris,
>> >
>> > On Thu, 2008-05-15 at 09:52 -0600, Chris Worley wrote:
>> >> Is there any command line utility to tell nodes that don't see the
>> >> route change to "go ask the SM again for your routes"... or "clear the
>> >> route table"?
>> >
>> > I'm not sure what you're asking. There is no route table at end nodes;
>> > only switch nodes and the SM maintains these. The end node only has path
>> > records which it has retrieved and perhaps cached. Path records should
>> > be refreshed when SM or local LID changes which are local events to the
>> > end node.
>>
>> After an sm change (i.e. using the "-r" switch),
>
> That should be a local LID change.
>
>>  nodes can't ping each
>> other over IPoIB (other protocols also can't communicate).
>
> Sounds like ULP issue(s) in handling this. What kernel and/or OFED
> version are you running ?

Currently, the SM is running OFED 1.3 on an RHEL4 2.6.9-67.0.4 kernel
with Lustre 1.6.4.2 changes.

The compute nodes are running the same kernel w/ OFED 1.2.5.5... which
will be upgraded to 1.3 by the end of the day.

Chris
>
> -- Hal
>
>> Restarting the OFED stack works, but modules won't unload if there was
>> something active (i.e. Lustre), so the only recource to getting the
>> OFED stack working again is a hard reboot.
>>
>> That's what I'd like to avoid if possible.
>>
>> Chris
>> >
>> > -- Hal
>> >
>> >> On Thu, May 15, 2008 at 5:19 AM, Sasha Khapyorsky <sashak at voltaire.com> wrote:
>> >> > Hi Hal,
>> >> >
>> >> > On 04:19 Mon 12 May     , Hal Rosenstock wrote:
>> >> >>
>> >> >> I filed this as bug 1031:
>> >> >> https://bugs.openfabrics.org/show_bug.cgi?id=1031
>> >> >>
>> >> >> > It would be nice if I could reproduce it in simulation.
>> >> >>
>> >> >> Yes, that would be nice; but I don't have a sim case.
>> >> >
>> >> > Do you have ibnetdiscover file for this case? If not from where report
>> >> > is coming?
>> >> >
>> >> > Sasha
>> >> > _______________________________________________
>> >> > general mailing list
>> >> > general at lists.openfabrics.org
>> >> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>> >> >
>> >> > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>> >> >
>> >> _______________________________________________
>> >> general mailing list
>> >> general at lists.openfabrics.org
>> >> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>> >>
>> >> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>> >
>> >
>
>