[openib-general] OSM: Using lid matrices in ucast manager

[Hal Rosenstock]

Hi Yevgeny & Sasha,

On Mon, 2006-12-18 at 20:30, Sasha Khapyorsky wrote:
> Hi Yevgeny,
> 
> On Tue, 2006-12-19 at 01:33 +0200, Yevgeny Kliteynik wrote:
> > Hi Hal.
> > 
> >  
> > 
> > I have a question about some patch that I want to send regarding lid
> > matrices usage in osm ucast
> > 
> > manager:
> > 
> >  
> > 
> > The FatTree routing doesn’t use the min hop tables, so we can skip the
> > lid matrices building in OSM.
> 
> The lid matrices are used in mcast_mgr for multicast routes generation.

Good point but fat tree seems to work for multicast (at least in my
subnet). How could that be ?

-- Hal

> > However, uca-st manager uses these lid matrices also to get the max lid
> > that is accessible from each
> > 
> > switch, which defines the LTF table size.
> > 
> > This max lid is obtained by calling osm_switch_get_max_lid_ho()
> > function, which in turn, calls 
> > 
> > osm_lid_matrix_get_max_lid_ho() for the switch’s lid matrix.
> > 
> > If the lid matrices weren’t built, then the
> >  osm_switch_get_max_lid_ho() function will return 0xFFFF,
> > 
> > and eventually osm will crash.
> > 
> >  
> > 
> > Of course, I don’t want to build all the lid matrices just to know the
> > max lid, so here’s what I’ve done:
> > 
> >  
> > 
> >       * I added a field to the osm_switch_t object: max_lid_ho (with
> >         default value 0xFFFF, should it 
> >         be 0x0 instead?).
> 
> Good thing. 0 is fine as default value IMHO.
> 
> >       * Added and three osm_switch_t methods for this new field:
> >         getter, setter, and is_set that returns
> >         true if this field has been set.
> 
> Why those methods? Everything you need is to access structure field and
> 'if (sw->max_lid_ho)' for "is_set" checks.
> 
> >       * The original osm_switch_get_max_lid_ho() has been updated to
> >         return this field value if it’s set.
> >       * Then in FatTree routing I set this field for each switch (I
> >         get the max lid ‘for free’ as a byproduct
> >         of the algorithm).
> >       * Now everything in the ucast manager works fine, except for the
> >         following two dump functions:
> >                 __osm_ucast_mgr_dump_ucast_routes (it uses hops)
> >                 ucast_mgr_dump_lid_matrix (obviously…)
> >         These two functions check at the beginning whether the
> >         max_lid_ho was set (using the ‘is_set’
> >         method), and return w/o printing anything if the answer is
> >         yes.
> > 
> >  
> > 
> > This way any other routing engine that uses lid matrix is not affected
> > by this change, and any routing 
> > 
> > engine that doesn’t use the lid matrix has a way to set the max lid
> > per switch explicitly.
> 
> Hope you are adding this for existing code.
> 
> > This approach works great, but I have a feeling that this is kinda
> > hack…
> 
> Moving max_lid(_ho) to switch structure looks like a good idea for me
> regardless to lid matrix build elimination.
> 
> The only problem I can see with lid matrices is mcast_mgr which uses
> this.
> 
> Sasha
> 
> > 
> >  
> > 
> > What do you think about this solution?
> > 
> > Any other suggestions?
> > 
> >  
> > 
> > Anyway, just wanted to hear your opinion before sending the patch.
> > 
> >    
> > 
> > Regards,
> > 
> >  
> > 
> > Yevgeny Kliteynik
> > 
> >  
> > 
> > Mellanox Technologies LTD
> > 
> > Tel: +972-4-909-7200 ext: 394
> > 
> > Fax: +972-4-959-3245
> > 
> > P.O. Box 586 Yokneam 20692 ISRAEL 
> > 
> >  
> > 
> >