[Users] does opensm support directed torus topology?

[Hal Rosenstock]

On Thu, Jun 7, 2018 at 1:37 PM, Alexandre Malotchko <a.malotchko at gmail.com>
wrote:

> >  What is the motivation
> > for this topology ? Why not just use a switch (all 27 hosts can be
> supported
> > on a single switch) or 2 (parallel non overlapping subnets) if you want
> > redundancy ?
>
> CA-to-CA latency is lower than CA-IBrouter-CA. Plus lower costs
> (avoids using specialized IB switch/router hardware).
>
> > Each CA-CA link is a separate subnet and would need to run at least 1 SM.
> > Some care would need to be taken to configure the SMs appropriately
> > (different subnet prefixes, more than 1 SM on a host). OpenSM supports
> > static routing between the different IB subnets via prefix_routes_file
> > option where each line is a prefix followed by GUID separated by white
> > space. You can find out more in the opensm man page under PREFIX ROUTES.
> > Issue here is that GUID must be that of a router port and the CAs are
> going
> > to advertise themselves as CA not router nodes. The CAs in this scenario
> are
> > both CAs and routers. Some way of indicating that to SM is needed (I know
> > how to do this from a spec perspective but it depends on actual
> > implementation - are some "off the shelf" CAs being used ?) and some
> > relatively minor changes to support that.
>
> We have tried to do a 3 (and 4) host topology with direct CA-to-CA
> links using dual CA IB cards.
>
> Running multiple instances of OpenSM running is an issue. From memory,
> the temp tables SM writes are overwritten from another SM instance.
> If I remember correctly, that issue was already raised but not addressed.
>

There are a number of installations successfully doing this. Separate
options files are needed as well as setting a couple of environment
variables otherwise the SM instances will collide on various files.


> I do not recall if we did resolve the temp tables issue, but in the
> end were unable to get it up and running on either Linux or windows.
> We did use subnet_prefix etc.
>
> In fact, it would help if someone could provide a working and tested
> configuration for the above.
>

One example is
http://lists.openfabrics.org/pipermail/ewg/2015-February/018341.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20180607/f9b80fac/attachment.html>