<html><body>
<p>Roland,<br>
<br>
Thanks for your quick response.<br>
<br>
Even SM supports 1000 MCGs, it's still not sufficitent for 250 nodes cluster, each node have 4 links for IPv6 without any scope/global IPv6 address configured.(250*4+ a few default MCGs) There will be a MCG overflow problem anyway in IPv6oIB.<br>
<br>
Thanks<br>
Shirley Ma<br>
IBM Linux Technology Center<br>
15300 SW Koll Parkway<br>
Beaverton, OR 97006-6063<br>
Phone(Fax): (503) 578-7638<br>
<br>
<br>
<img width="16" height="16" src="cid:1__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt="Inactive hide details for Roland Dreier <rdreier@cisco.com>">Roland Dreier <rdreier@cisco.com><br>
<br>
<br>
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr valign="top"><td style="background-image:url(cid:2__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com); background-repeat: no-repeat; " width="40%">
<ul>
<ul>
<ul>
<ul><b><font size="2">Roland Dreier <rdreier@cisco.com></font></b><font size="2"> </font>
<p><font size="2">02/16/2007 09:00 AM</font></ul>
</ul>
</ul>
</ul>
</td><td width="60%">
<table width="100%" border="0" cellspacing="0" cellpadding="0">
<tr valign="top"><td width="1%"><img width="58" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<div align="right"><font size="2">To</font></div></td><td width="100%"><img width="1" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<font size="2">Shirley Ma/Beaverton/IBM@IBMUS</font></td></tr>
<tr valign="top"><td width="1%"><img width="58" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<div align="right"><font size="2">cc</font></div></td><td width="100%"><img width="1" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<font size="2">"Michael S. Tsirkin" <mst@mellanox.co.il>, openib-general@openib.org</font></td></tr>
<tr valign="top"><td width="1%"><img width="58" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<div align="right"><font size="2">Subject</font></div></td><td width="100%"><img width="1" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""><br>
<font size="2">Re: IPv6oIB neighbour discover broken when MCGs overflow</font></td></tr>
</table>
<table border="0" cellspacing="0" cellpadding="0">
<tr valign="top"><td width="58"><img width="1" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""></td><td width="336"><img width="1" height="1" src="cid:3__=08BBF817DFCD7DA78f9e8a93df938@us.ibm.com" border="0" alt=""></td></tr>
</table>
</td></tr>
</table>
<br>
<tt> > We have a customer issue regarding IPv6oIB. In the subnet, there are<br>
> limited number of MCGs supported. So when there are multiple IPv6 addresses<br>
> are assigned to one interface, each IPv6 address will have one unique<br>
> solicited-node address (depends on their groupID). Then in a large subnet,<br>
> we will have tons of MCGs. If IPv6 solicited node addresses exceed the<br>
> number of MDGs in this subnet, then IPv6 neighbour discovery will be<br>
> broken, this won't happen in Ethernet since sendonly doesn't require sender<br>
> to be joined any MCG.<br>
<br>
> I have done an initial patch to addresss MCG overflow problem and redirect<br>
> the solicited-node address to all hosts node address, thus IPv6 neighbour<br>
> discovery will work no matter how many IPv6 addresses in this subnet. This<br>
> patch is only triggered with IPv6 enabled and MGC overflows, so there is<br>
> almost no performance penalty.<br>
<br>
I really don't like this approach, since it can break things in very<br>
subtle ways (eg suppose one node fails to join its solicited node<br>
group, but then a later node wants to talk to it and succeeds in<br>
joining the solicited node group as a send-only member -- since the<br>
first node is not a member then it will never see the ND messages).<br>
<br>
I much prefer to fix the SM not to impose too-low limits on the number<br>
of MCGs. Supporting O(# nodes) MCGs is really not a very onerous<br>
requirement on the SM.<br>
<br>
- R.<br>
</tt><br>
</body></html>