[ofa-general] Re: openSM: Different IB MTUs

[Eitan Zahavi]

Hi Shirley,
 
I think I understand where your question comes from...
Many have issue with heterogonous fabrics where not all nodes have same
MTU or Speed.
Especially when IPoIB relies on all nodes joining the broadcast group.
 
The term "join" for multicast groups is a little overloaded.
If a node joins an existing MC group it has to have a rate (speed *
width) > MCG.rate and support MTU > MCG.MTU otherwise it is denied.
If the join is actually a "create" the node has to provide the rate and
MTU which define the MCG values.
 
To allow for administrator to control the IPoIB MCGs MTU and rate OpenSM
provides the means to control these
values per partition. See the doc/partition-config.doc 
Still the administrator should know what would be the lowest MTU and
rate the nodes expected to join the IPoIB subnet have.
The tradeoff is in the hands of the administrator who can set a value
that will prevent slow nodes from joining the group, 
or assign a low value that will fit all nodes but slow down
communication ...
 
EZ

Eitan Zahavi 
Senior Engineering Director, Software Architect 
Mellanox Technologies LTD 
Tel:+972-4-9097208
Fax:+972-4-9593245 
P.O. Box 586 Yokneam 20692 ISRAEL 

 


________________________________

	From: general-bounces at lists.openfabrics.org
[mailto:general-bounces at lists.openfabrics.org] On Behalf Of Hal
Rosenstock
	Sent: Wednesday, July 25, 2007 10:01 PM
	To: Shirley Ma
	Cc: general at lists.openfabrics.org
	Subject: [ofa-general] Re: openSM: Different IB MTUs
	
	
	Shirley,
	
	
	On 7/25/07, Shirley Ma <xma at us.ibm.com> wrote: 

		Hal,
		
		Thanks for your prompt reply. I am asking for how openSM
handle different link MTUs in SA MCMemberRecord MTU. For example, if we
have some links MTU as 2K, some links MTU as 1K. Then when enabling
IPoIB, how does SM decide IPoIB broadcast group MCMemberRecord MTU size?
When creating an IB multicast group from a 2K MTU node first, which PMTU
value is attaching to this IB multicast group MCMemberRecord MTU? 

	 
	MCMemberRecord MTU gets the group MTU (when created). This is
either this first joiner with sufficient components or preconfigured
(and MTU can be set in the config). If a joiner has insufficient MTU for
the group, it is denied. 
	 
	-- Hal
	 


		Thanks
		Shirley Ma
		
		 "Hal Rosenstock" < hal.rosenstock at gmail.com>
		
		
		

				"Hal Rosenstock" <
hal.rosenstock at gmail.com> 

				07/25/07 10:57 AM

 

To

Shirley Ma/Beaverton/IBM at IBMUS	


cc

general at lists.openfabrics.org	


Subject

Re: openSM: Different IB MTUs	
	 	
		
		Shirley,
		
		On 7/25/07, Shirley Ma < xma at us.ibm.com
<mailto:xma at us.ibm.com> > wrote: 

			Hello Hal,
			
			How does openSM handle CAs with different MTUs
in the same subnet? For example, IPoIB broadcast group MTU, IB multicast
group PMTU? Does openSM pick up the smallest MTU in the subnet? 


		
		Are you asking about link MTU, SA
PathRecord/MultiPathRecord MTU, SA MCMemberRecord MTU, or all of these ?
		
		-- Hal 
		

			Thanks
			Shirley Ma


		

		

		



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070725/c381ef85/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: graycol.gif
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070725/c381ef85/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: ecblank.gif
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070725/c381ef85/attachment-0001.gif>