[ofa-general] IPoIB Bonding with OpenSM

[Amar Mudrankit]

I was testing the IPoIB failover/failback using the bonding mechanism with
Open SM running in the IB subnet. I observed that the failover does not
reliably occur IB port is made down using "ibportstate" command.

The test steps I followed and test configuration is as follows :

Pings to an IPoIB destination were started over the bond0 interface(which is
configured as mentioned below). Pings continue properly. Failover to ib1
does not occur when I disconnect port 1 (corresponding to ib0)  using

$ ibportstate disable

command.

In log, I can see the messages

kernel: bonding: bond0: link status definitely down for interface ib0,
disabling it
kernel: bonding: bond0: making interface ib1 the new active one.

But, the pings stop. Also, I noticed the process status which shows :

PPID   PID  PGID   SID TTY      TPGID STAT   UID   TIME COMMAND
  61  2503     1     1 ?           -1 D<       0   0:00 [ib_inform]
  61  2504     1     1 ?           -1 D<       0   0:00 [local_sa]

Is this expected ?

/etc/infiniband/openib.conf

ONBOOT=yes
UCM_LOAD=no
RDMA_CM_LOAD=yes
RDMA_UCM_LOAD=yes
RENICE_IB_MAD=no
MTHCA_LOAD=yes
IPOIB_LOAD=yes
SET_IPOIB_CM=yes
SDP_LOAD=yes
SRP_LOAD=no
SRPT_LOAD=no
RDS_LOAD=no
SRPHA_ENABLE=no

IPOIBBOND_ENABLE=yes
IPOIB_BONDS=bond0
bond0_IP=100.1.1.13
bond0_SLAVES=ib0,ib1

Source IPoIB m/c (bonding enabled) : OFED-1.3-rc4, RHEL5, MT25208
Destination IPoIB m/c             : OFED-1.3-rc4, SLES10, MT25208

I am pinging the IPoIB interface over a machine which is running OpenSM.

Has somebody tested this kind of scenario ever or I am missing something?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080214/301f5085/attachment.html>