[openib-general] ib_gid_is_link_local

[Jason Gunthorpe]

On Fri, Jan 05, 2007 at 11:44:49AM -0500, Hal Rosenstock wrote:

> > However, it might be smart to have opensm consider the routers to be a
> > send-only member for every MLID..
> 
> Do you mean non-member rather than send-only member ? Routers need to
> receive as well as send, right ? Or are you worried about some other
> issue here ?

I would like it if routers did not have to worry about joins in order
to send a multicast packet. There really isn't a good way to know how
long to keep a join active for.. Having them be send-only members of
every relevent group skips that problem and decreases the latency for
first-packet multicast forwarding.
 
> Also, I'm still not sure about a couple of aspects of "every MLID":
> 1. Wouldn't the router only want to be full member of link local scoped
> MGIDs (that it was interested in locally) ? Are you saying any local
> scoped MGIDs not of interest would just get dropped anyhow ? If that is
> the point here, that would work but isn't there a performance impact of
> doing so ? 
> 2. Similarly for any other (non local scope) MGRPs which do not match
> across any router ports, isn't there a performance impact of receiving
> and then having to drop/filter these packets ?

I think there is a balance to be had here, on one side if you have
alot of multicast groups (ie Ipv6 SNMs) then requiring alot of
extra work to keep the SM informed about what is going on is more
harmful than having the router get more multicast traffic than it
optimally could.

A router must already keep track of what multicast groups are
forwarded to what ports, so it is virtually free for it to also do
filtering.

[Aside: The more I think about scaling a router up the more it seems
 to me that the router and SM need alot of intercommunication. The
 most efficient thing would be if the router could maintain a replica
 of the entire SM database for paths and multicast.
 The router would then always be ready to handle any incoming
 packet, just like an IB switch.]

> Right, the router is some sort of member on the MGRPs "of interest". I
> think you are trying to make that list of MGRPs "of interest" simpler
> and utilize filtering where not needed (as I mentioned above), but I may

Yes.

Simpler, I hope :>

> > A onlink line routing table just terminates the routing
> > lookup. 'unreachable' is another termination. A via line changes the
> > next hop GID and creates more lookups until an onlink is reached.
> 
> So is the specification of all multicast as onlink a short term thing
> then ?
> 
> Also, with using onlink for all multicast, is there some forwarding
> determination made somewhere in the router stack ?

I think it is useful to keep the router stack and the SM stack
seperate.  Especailly when it comes to multicast. The router will have
a multicast routing table that works somewhat differently than the
unicast table. This table would indicate which ports in the router are
part of each group. The SM should only need a MGID to MLID path
translation. This is similar to the distinction between a host routing
table and a router routing table in IP land - where hosts generally do
not have multicast routing information.

> Yes, I have no idea how IPv6 will work with large inter subnet clusters
> either. We had a thread on this a while ago and I think it died out at
> that point. To state the obvious, I think some changes need to be made
> for IPv6 to work well with current IB hardware or perhaps some
> configuration restrictions ?

Yes, I agree. I wonder if the IPoIB RFC authors considerd the negative
impact of IPv6 SNM when they designed the specification? It would be
much better if 1 IP subnet = 1 IB subnet.

> > Yes, but in this case I don't think multicast routing can be pushed to
> > the host. It is either the router or some combination of the router
> > and the SM.
> 
> I'm not quite following you on this yet. Why/how is host multicast
> routing any different (than unicast) ?

Well, I can't see how to make this situation sane if the host is in
control and the routers/sm are fairly passive:
- Two subnets, each with nodes joined to multicast group M
- Two routers connecting the two subnets (multipath)
- Each host has an inter-subnet multicast spanning tree and knows
  which router to use for M
- Host sends a packet for M, what LRH does it use?

Jason