[openib-general] ib_gid_is_link_local

[Jason Gunthorpe]

On Tue, Jan 09, 2007 at 01:17:47AM -0500, Hal Rosenstock wrote:

> > I would like it if routers did not have to worry about joins in order
> > to send a multicast packet.
> 
> Send-only members are not supposed to receive. How do they receive then
> ? Send-only members do not receive. Don't routers need to receive
> multicast (as well as send) ?

How about more exactly: The SM could implicity consider the router as
a send-only member of all forwardable multicast groups until a reason
arises for it to be a full member.

Anyhow, I think this discussion has lost context and we are not
thinking about the same things. Let me describe to you how I think
that a router today can implement multicast without special SM
support using IBA defined protocols:

Let me try to do that:
- The router maintains a table of all forwardable multicast groups on
  each IB subnet that it is connected to.
- It also tracks for each router port which groups have receivers
  on the local subnet. If so the group on that port is flagged
  'rxer' otherwise 'txonly'
- This table is kept in sync with the SM by using SM traps and SM queries.
- Each router port then computes a set of joins to perform on the local
  subnet based on this table:
   Join Type    Local_MGID   Remote_MGID
    none         none         txonly
    none         none         rxer
    none         txonly       none
    none         txonly       txonly        [No receiver]
    full         txonly       rxer          [Only remote receiver]
    none         rxer         none
    send-only    rxer         txonly        [Only local receiver]
    full         rxer         rxer          [Both receiver]

  Remote_MGID would be rxer if any other participating port
  has a rxer flag for this MGID. [participating port being
  derived from a multicast routing protocol].

  (How exactly to determine the rxer/txonly flag and if this
   optimization is even really necessary is not something I have spent
   alot of time on just yet - but this conceptually describes
   the optimal, minimum spanning methodology.)
- The router connects to other routers on the local subnet and
  performs a multicast routing protocol to produce a inter-subnet
  multicast spanning tree for each MGID. The results from this control
  which ports participate in each MGID.
- Finally, the router programs its internal forwarding path.

As an example using IPv6 SNM:
1) A new nodes comes up on subnet alpha and registers SNM MGID A as
   full membership.
2) The subnet alpha local router port learns of #1 from the local SM.
3) The router forwards the new mgid to other routers it is connected
   to via the multicast routing protocol.
4) On the beta subnet, another node registers as send-only for SNM MGID
   A.
5) The beta local router port learns of #4 from the local SM.
6) The router inspects its MGID table and finds one of its ports
   has a path to the rxer in #1. It joins MGID A on subnet beta as
   a full member, and the other port joins MGID A as a send-only
   member.
7) The above repeats through the chain of subnets until subnet alpha
   is reached.
8) A port on the router connected to subnet alpha sees the MGID A
   creation on one of its other ports and registers as send-only
   for MGID A on subnet alpha. (similar to step #6)
9) The host sends the SNM, unsubscribres from MGID A and the process
   reverses itself.

I think this is within what IBA already defines and is pretty much
what has to be done today to have a chance of working with existing
subnet managers. I don't think it needs changes to the SM. I
don't think it scales very well since it requires alot of exchanges
between the SM and the routers. This is more or less what I had in
mind during the concall we had last year...

Also, I expect the first SNM message will be lost since the SM will
ack the host before the router has received the trap, found the new
MGID and joined it. (I don't think that is very good :<)

======

The above describes the router as being autonomous of the SM. The
routers learn of data the SM has through queries (a pull
model). Another approach is to have the SM program the routers
explicitly (a push model). In this view the router is more like an IB
switch from a SM programming perspective. It has a more complex
LinearFDB that uses GIDs rather than LIDs and a more complex multicast
table that works on MGIDs rather than MLIDS. Like a switch the SM
would program the router as needed.

I view this as being more in line with the IB treatment of the network
as a completely managed resource. It should be more efficient since
the SM only sends what changes to the routers rather than the routers
responding to traps/etc. I'd ultimately like to find other interested
people to work on this idea since I think it has merit..

It is with this second case where my prior thoughts about optimization
strategies make more sense. (Ie pre-arranging send-only status for the
router is an optimization that lets the SM do less work on group
creation, the SM can 'replicate' the SM database into the router in
the same way it 'replicates' it into the switches, etc)

Regards,
Jason