[Users] does opensm support directed torus topology?

[Hal Rosenstock]

On Thu, Jun 7, 2018 at 8:01 AM, John Evan <evajohn.777 at gmail.com> wrote:

> yes, all the hosts don't have direct connection with all the other hosts.
>
> Then assuming that the CA's have switch functions (forwarding the packets)
> in them , whether opensm can support the topology?
>

I largely agree with what Peter wrote but I'll take my cut at explaining
this:

Each CA-CA link is a separate subnet and would need to run at least 1 SM.
Some care would need to be taken to configure the SMs appropriately
(different subnet prefixes, more than 1 SM on a host). OpenSM supports
static routing between the different IB subnets via prefix_routes_file
option where each line is a prefix followed by GUID separated by white
space. You can find out more in the opensm man page under PREFIX ROUTES.
Issue here is that GUID must be that of a router port and the CAs are going
to advertise themselves as CA not router nodes. The CAs in this scenario
are both CAs and routers. Some way of indicating that to SM is needed (I
know how to do this from a spec perspective but it depends on actual
implementation - are some "off the shelf" CAs being used ?) and some
relatively minor changes to support that.

The second issue is that a router like forwarder would need to be added for
packets sent to CA but not terminated there(DGID is not one of CA GIDs but
it was addressed to CA LID). I don't recall off the top of my head nor did
I go through the spec to see how CAs handle such packets but AFAIR there is
no DGID validation on receive as I recall from adding in the SA well known
GID support but there maybe something I'm forgetting here. So assuming
that's the case (more diligence should be spent on proving this), such
packets can be sent and received in user space and a (software) forwarder
could be implemented for these but is there a performance requirement ? The
forwarder would need a routing table and I suppose that would also be
configured somehow.

All of the above (assuming the performance of a software based forwarder is
acceptable) seems like a lot of development to me. What is the motivation
for this topology ? Why not just use a switch (all 27 hosts can be
supported on a single switch) or 2 (parallel non overlapping subnets) if
you want redundancy ?

-- Hal


>
> --Jon E
>
>
> On Thu, Jun 7, 2018 at 5:28 PM, John Evan <evajohn.777 at gmail.com> wrote:
>
>> yes, all the hosts don't have direct connection with all the other hosts.
>>
>> Then assuming that the CA's have switch functions (forwarding the
>> packets) in them , whether opensm can support the topology?
>>
>> --Jon E
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/users/attachments/20180607/940ba5e9/attachment.html>