[ofa-general] ipoib bonding problems in 1.3-beta2 and 1.2.5.4,

[Vu Pham]

>
>
> Please note that in RH5 there's a native support for bonding 
> configuration through the initscripts tools (network scripts, etc), 
> see section 3.1.2 at the ib-bonding.txt document provided with the 
> bonding package.
>

Hi Or,

Thanks for the pointer

>> I moved our systems back to ofed-1.2.5.4 and tested ib-bond again. We 
>> tested it with ib0 and ib1 (connected to different switch/fabric) 
>> been on the same subnet (10.2.1.x, 255.255.255.0) and on different 
>> subnets (10.2.1.x and 10.3.1.x, 255.255.255.0). In both cases there 
>> is the issue of loosing communication between the servers if nodes 
>> have not been on the same primary ib interface.
>
> Generally speaking, I don't see the point in using bonding for 
> --high-availability-- where each slave is connected to different 
> fabric. This is b/c when there's fail-over in one system you need also 
> the second system to fail-over, you would also not be able to count on 
> local link detection mechanisms, since the remote node also must 
> fail-over now even with his local link being perfectly fine. This is 
> correct regardless of the interconnect type.
>
> Am I missing something here regarding to your setup?
>
> The question on usage case of bonding over separate fabrics have been 
> brought to me several times and I gave this answer, no-one ever tried 
> to educate me why its interesting, maybe you will do so...
>

I don't have good reason. I used two separated fabrics configuration 
because my lacking understanding on ethernet/ib bonding and the old 
methodology way of redundancy in ethernet  & FC using two separated fabrics.

> Also what do you mean with "ib0 and ib1 been on the same/different 
> subnets" its only the master device (eg bond0, bond1, etc) with has 
> association/configuration with an IP subnet, correct?

I talked about ib0,ib1 subnets because I set up bonding using 
openib.conf and openibgd. I understand now we don't need to setup ib0, 
ib1 using distribution initscript to setup bonding.

thanks for your explanation

-vu