[ofa-general] Re: ipoib / bonding and OFED

[Scott Weitzenkamp (sweitzen)]

Bob, it is now possible to configure IPoIB bonding in
/etc/infiniband/openib.conf, this configuration file includes the
following boilerplate.

# Enable the bonding driver on startup
IPOIBBOND_ENABLE=no
# Set bond interface names
#IPOIB_BONDS=bond0,bond1
# Set specific bond params; address and slaves
#bond0_IP=10.10.10.1
#bond0_SLAVES=ib0,ib1
#bond1_IP=20.10.10.1
#bond1_SLAVES=ib2,ib3,ib4

Scott Weitzenkamp
SQA and Release Manager
Server Virtualization Business Unit
Cisco Systems
 

> -----Original Message-----
> From: general-bounces at lists.openfabrics.org 
> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Or Gerlitz
> Sent: Tuesday, May 29, 2007 12:56 AM
> To: Bob Kossey
> Cc: OpenFabrics General
> Subject: [ofa-general] Re: ipoib / bonding and OFED
> 
> Bob Kossey wrote:
> > I copied OR since I think this is related to his OFED HA work, and
> > he might have some insights.  A few more questions for Or:
> > I was trying to use ipoib bonding with OFED 1.2 rc2 and a 
> 2.6.9 kernel,
> > but was not able to get it to work so far.  I saw your 
> Sonoma bonding
> > slides, and you mention kernel bonding driver changes were needed.
> > 2. Is there a minimum kernel version, with the kernel bonding driver
> > changes, that is required to use bonding with OFED ipoib?
> 
> Just to have a base line here: to get bonding to work with IPoIB, you 
> should use the bonding driver provided with OFED 1.2. This 
> driver is the 
>   upstream one (of 2.6.20) being patched to support IPoIB and 
> backported 
> to RH5, SLES10 and RH4 U3/4/5, other kernels are not supported.
> 
> If you were using the ofed bonding on a system that matches 
> the support 
> matrix it should worl. If do have problems under this config, please 
> either open a bug at the ofed bugzilla
> @ bugs.openfabrics.org assigned to monis at voltaire.com (Moni Shoua) or 
> send first report/question to Moni and CC ewg at lists.openfabrics.org
> 
> Please note that between RC2 and RC4 (to be released today etc) some 
> bugs were fixed, you can search in the bugzilla to see what.
> 
> > 3. The bonding driver uses the HWADDR from the underlying ipoib
> > devices, how does it obtain the HWADDR?  Does it use the 
> full 20 bytes,
> > or some subset?
> 
> when enslaving IPoIB devices, the bonding driver uses the full hw 
> address of the active slave, it simply looks on the dev_addr field of 
> the slave struct netdevice (see include/linux/netdevice.h)
> 
> > 4. What use_carrier options for link status detection does 
> OFED ipoib 
> > support,
> > MII, ETHTOOL or netif_carrier_ok?
> 
> the mii/ethertool etc local link detection methods of the 
> bonding driver 
>   are somehow deprecated, since nowadays almost any network device 
> support the netif_carrier_ok call. The --default-- of the upstream 
> bonding driver (eg the one we use in OFED and the 2.6.21 
> listed below) 
> is to set the use_carrier mod param to 1 that is mii is not 
> used anymore.
> 
> > author:         Thomas Davis, tadavis at lbl.gov and many others
> > description:    Ethernet Channel Bonding Driver, v3.1.2
> > version:        3.1.2
> > parm:           use_carrier:Use netif_carrier_ok (vs MII 
> ioctls) in miimon; 0 for off, 1 for on (default) (int)
> > parm:           miimon:Link check interval in milliseconds (int)
> 
> > If you have any good examples of bonding configuration 
> settings that work
> > with OFED, I'd appreciate that also.
> 
> The bonding RPM provided with OFED is made of a driver, 
> script and some 
> help text containing usage examples, please take a look there 
> and let me 
> know if you have further questions.
> 
> > $ rpm -ql ib-bonding-0.9.0-2.6.9_42.ELsmp
> > 
> /lib/modules/2.6.9-42.ELsmp/updates/kernel/drivers/net/bonding
> /bonding.ko
> > /usr/bin/ib-bond
> > /usr/share/doc/ib-bonding-0.9.0/ib-bonding.txt
> 
> The ofed service (/etc/init.d/openibd) was enhanced to allow for 
> --persistent-- bonding configuration, please see the bonding 
> section at
> docs/ipoib_release_notes.txt to see how to do it.
> 
> Or.
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>