[openib-general] ib_gid_is_link_local

[Hal Rosenstock]

On Tue, 2007-01-09 at 00:22, Jason Gunthorpe wrote:
> On Fri, Jan 05, 2007 at 11:44:49AM -0500, Hal Rosenstock wrote:
> 
> > > However, it might be smart to have opensm consider the routers to be a
> > > send-only member for every MLID..
> > 
> > Do you mean non-member rather than send-only member ? Routers need to
> > receive as well as send, right ? Or are you worried about some other
> > issue here ?
> 
> I would like it if routers did not have to worry about joins in order
> to send a multicast packet.

Send-only members are not supposed to receive. How do they receive then
? Send-only members do not receive. Don't routers need to receive
multicast (as well as send) ?

> There really isn't a good way to know how long to keep a join active for..

What about MGID creation/deletion events ?

> Having them be send-only members of every relevent group 

How does the router know "every relevant" group ?

> skips that problem and decreases the latency for
> first-packet multicast forwarding.

It could be a benefit on first packet MC forwarding on a new group but
it depends on when the first packet is received relative to the group
detected and joined.

> > Also, I'm still not sure about a couple of aspects of "every MLID":
> > 1. Wouldn't the router only want to be full member of link local scoped
> > MGIDs (that it was interested in locally) ? Are you saying any local
> > scoped MGIDs not of interest would just get dropped anyhow ? If that is
> > the point here, that would work but isn't there a performance impact of
> > doing so ? 
> > 2. Similarly for any other (non local scope) MGRPs which do not match
> > across any router ports, isn't there a performance impact of receiving
> > and then having to drop/filter these packets ?
> 
> I think there is a balance to be had here, on one side if you have
> alot of multicast groups (ie Ipv6 SNMs) then requiring alot of
> extra work to keep the SM informed about what is going on is more
> harmful than having the router get more multicast traffic than it
> optimally could.
> 
> A router must already keep track of what multicast groups are
> forwarded to what ports, so it is virtually free for it to also do
> filtering.
> 
> [Aside: The more I think about scaling a router up the more it seems
>  to me that the router and SM need alot of intercommunication. The
>  most efficient thing would be if the router could maintain a replica
>  of the entire SM database for paths and multicast.
>  The router would then always be ready to handle any incoming
>  packet, just like an IB switch.]

There is no IBA standard for replicating the SM or SA database. This is
a similar issue which multiple SMs in the same subnet might have
depending on the "approach" taken for this.

> > Right, the router is some sort of member on the MGRPs "of interest". I
> > think you are trying to make that list of MGRPs "of interest" simpler
> > and utilize filtering where not needed (as I mentioned above), but I may
> 
> Yes.
> 
> Simpler, I hope :>
> 
> > > A onlink line routing table just terminates the routing
> > > lookup. 'unreachable' is another termination. A via line changes the
> > > next hop GID and creates more lookups until an onlink is reached.
> > 
> > So is the specification of all multicast as onlink a short term thing
> > then ?
> > 
> > Also, with using onlink for all multicast, is there some forwarding
> > determination made somewhere in the router stack ?
> 
> I think it is useful to keep the router stack and the SM stack
> seperate.

Yes, but on the other hand, you just said you wanted a partial copy of
the SM database for the router...

> Especailly when it comes to multicast. The router will have
> a multicast routing table that works somewhat differently than the
> unicast table. This table would indicate which ports in the router are
> part of each group. The SM should only need a MGID to MLID path
> translation.

Doesn't it already have this ?

>  This is similar to the distinction between a host routing
> table and a router routing table in IP land - where hosts generally do
> not have multicast routing information.

I don't think the SM needs the multicast routing (intersubnet)
information either.

> > Yes, I have no idea how IPv6 will work with large inter subnet clusters
> > either. We had a thread on this a while ago and I think it died out at
> > that point. To state the obvious, I think some changes need to be made
> > for IPv6 to work well with current IB hardware or perhaps some
> > configuration restrictions ?
> 
> Yes, I agree. I wonder if the IPoIB RFC authors considerd the negative
> impact of IPv6 SNM when they designed the specification?

Not sure.

> It would be much better if 1 IP subnet = 1 IB subnet.

Yes, it would be better in terms of this but it was an architectural
goal to allow flexibility in the IP <-> IB subnet mappings. There was a
lot of discussion about this and there were earlier schemes which
restricted to that mapping.

> > > Yes, but in this case I don't think multicast routing can be pushed to
> > > the host. It is either the router or some combination of the router
> > > and the SM.
> > 
> > I'm not quite following you on this yet. Why/how is host multicast
> > routing any different (than unicast) ?
> 
> Well, I can't see how to make this situation sane if the host is in
> control and the routers/sm are fairly passive:
> - Two subnets, each with nodes joined to multicast group M
> - Two routers connecting the two subnets (multipath)
> - Each host has an inter-subnet multicast spanning tree and knows
>   which router to use for M
> - Host sends a packet for M, what LRH does it use?

I would think that all the host does is use the MLID for the group
obtained from it's join (or query). The IB routers need to work out
which one is the active forwarder for each group (via their multicast
routing protocol (DVMRP, MOSPF, PIM, ...). I have no clue how multipath
could work for multicast.

-- Hal

> Jason