[openib-general] IPv6oIB neighbour discover broken when MCGs overflow

[Roland Dreier]

 > We have a customer issue regarding IPv6oIB. In the subnet, there are
 > limited number of MCGs supported. So when there are multiple IPv6 addresses
 > are assigned to one interface, each IPv6 address will have one unique
 > solicited-node address (depends on their groupID). Then in a large subnet,
 > we will have tons of MCGs. If IPv6 solicited node addresses exceed the
 > number of MDGs in this subnet, then IPv6 neighbour discovery will be
 > broken, this won't happen in Ethernet since sendonly doesn't require sender
 > to be joined any MCG.

 > I have done an initial patch to addresss MCG overflow problem and redirect
 > the solicited-node address to all hosts node address, thus IPv6 neighbour
 > discovery will work no matter how many IPv6 addresses in this subnet. This
 > patch is only triggered with IPv6 enabled and MGC overflows, so there is
 > almost no performance penalty.

I really don't like this approach, since it can break things in very
subtle ways (eg suppose one node fails to join its solicited node
group, but then a later node wants to talk to it and succeeds in
joining the solicited node group as a send-only member -- since the
first node is not a member then it will never see the ND messages).

I much prefer to fix the SM not to impose too-low limits on the number
of MCGs.  Supporting O(# nodes) MCGs is really not a very onerous
requirement on the SM.

 - R.