[openib-general] IPv6oIB neighbour discover broken when MCGs overflow

[Shirley Ma]

Roland,

Thanks for your quick response.

Even SM supports 1000 MCGs, it's still not sufficitent for 250 nodes
cluster, each node have 4 links for IPv6 without any scope/global IPv6
address configured.(250*4+ a few default MCGs) There will be a MCG overflow
problem anyway in IPv6oIB.

Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638




                                                                           
             Roland Dreier                                                 
             <rdreier at cisco.co                                             
             m>                                                         To 
                                       Shirley Ma/Beaverton/IBM at IBMUS      
             02/16/2007 09:00                                           cc 
             AM                        "Michael S. Tsirkin"                
                                       <mst at mellanox.co.il>,               
                                       openib-general at openib.org           
                                                                   Subject 
                                       Re: IPv6oIB neighbour discover      
                                       broken when MCGs overflow           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           




 > We have a customer issue regarding IPv6oIB. In the subnet, there are
 > limited number of MCGs supported. So when there are multiple IPv6
addresses
 > are assigned to one interface, each IPv6 address will have one unique
 > solicited-node address (depends on their groupID). Then in a large
subnet,
 > we will have tons of MCGs. If IPv6 solicited node addresses exceed the
 > number of MDGs in this subnet, then IPv6 neighbour discovery will be
 > broken, this won't happen in Ethernet since sendonly doesn't require
sender
 > to be joined any MCG.

 > I have done an initial patch to addresss MCG overflow problem and
redirect
 > the solicited-node address to all hosts node address, thus IPv6
neighbour
 > discovery will work no matter how many IPv6 addresses in this subnet.
This
 > patch is only triggered with IPv6 enabled and MGC overflows, so there is
 > almost no performance penalty.

I really don't like this approach, since it can break things in very
subtle ways (eg suppose one node fails to join its solicited node
group, but then a later node wants to talk to it and succeeds in
joining the solicited node group as a send-only member -- since the
first node is not a member then it will never see the ND messages).

I much prefer to fix the SM not to impose too-low limits on the number
of MCGs.  Supporting O(# nodes) MCGs is really not a very onerous
requirement on the SM.

 - R.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070216/2f791386/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070216/2f791386/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pic12802.gif
Type: image/gif
Size: 1255 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070216/2f791386/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070216/2f791386/attachment-0002.gif>