[ofa-general] Re: [PATCH OpenSM 0/3] Fat Tree - Routing between non-CN nodes

[Sasha Khapyorsky]

Hi Nicolas,

On 14:42 Thu 18 Dec     , Nicolas Morey Chaisemartin wrote:
>
> We are current working on a Ftree topology where IO nodes are connected on 
> spine switches.
> Using the cn_guid_file and root_guid_file works great.
> It is possible to route the whole tree as a fat tree. All the CNs are 
> connected to the other CN and IO nodes.
> However, we are missing some connectivity between IO nodes. This is the 
> expected behavior as the route between those IO nodes would have
> to go down to go back up on another spine switch.
>
> However, we need at least a bit of connectivity between those nodes. There 
> won't be any real traffic but just some "ping" for HA purposes.
>
> Therefore, I have implemented two new options to openSM: io_guid_file and 
> max_reverse_hops.
> The io_guid_file provides a list of all the IO guid (it may differs from 
> the list of non-CN nodes)

"IO" is specific for your setup. Could we find more generic name for such
nodes?

> The max_reverse_hops gives the number of time IO nodes (described by 
> io_guid_file) are allowed to use a switch backward.

Don't those two options duplicate each others somehow? If we want to
connect io nodes anyway, why max_reverse_hops should be important?

Or probably instead of having io nodes guids list we prefer to connect
everything N hops from roots? Then sort of --connect-roots extension
(--connect-roots=3) could work. No?

>
> According to my tests this has absolutely no effects on regular routing and 
> manages to connect the io nodes together, if max_reverse_hops is big 
> enough.
>
> This is a first draft for this feature. I'd be happy to have some feedback 
> about how to upgrade it and make it as clean as possible, wether it is 
> integrated in the mainstream or not.

Since this functionality is optional, useful and shouldn't change a
default behavior it can be suitable for main stream IMO.

Sasha