[Users] does opensm support directed torus topology?

[Alexandre Malotchko]

>  What is the motivation
> for this topology ? Why not just use a switch (all 27 hosts can be supported
> on a single switch) or 2 (parallel non overlapping subnets) if you want
> redundancy ?

CA-to-CA latency is lower than CA-IBrouter-CA. Plus lower costs
(avoids using specialized IB switch/router hardware).

> Each CA-CA link is a separate subnet and would need to run at least 1 SM.
> Some care would need to be taken to configure the SMs appropriately
> (different subnet prefixes, more than 1 SM on a host). OpenSM supports
> static routing between the different IB subnets via prefix_routes_file
> option where each line is a prefix followed by GUID separated by white
> space. You can find out more in the opensm man page under PREFIX ROUTES.
> Issue here is that GUID must be that of a router port and the CAs are going
> to advertise themselves as CA not router nodes. The CAs in this scenario are
> both CAs and routers. Some way of indicating that to SM is needed (I know
> how to do this from a spec perspective but it depends on actual
> implementation - are some "off the shelf" CAs being used ?) and some
> relatively minor changes to support that.

We have tried to do a 3 (and 4) host topology with direct CA-to-CA
links using dual CA IB cards.

Running multiple instances of OpenSM running is an issue. From memory,
the temp tables SM writes are overwritten from another SM instance.
If I remember correctly, that issue was already raised but not addressed.

I do not recall if we did resolve the temp tables issue, but in the
end were unable to get it up and running on either Linux or windows.
We did use subnet_prefix etc.

In fact, it would help if someone could provide a working and tested
configuration for the above.