[ofa-general] RE: two interfaces with ipoib

[Scott Weitzenkamp (sweitzen)]

How is:

  sysctl -w net.ipv4.conf.all.arp_ignore=2 

different from:

  for i in /proc/sys/net/ipv4/conf/ib*/arp_filter; do echo 1 > $i; done

I have been using the latter successfully regarding this issue.

Scott Weitzenkamp
SQA and Release Manager
Server Virtualization Business Unit
Cisco Systems
 

> -----Original Message-----
> From: general-bounces at lists.openfabrics.org 
> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Or Gerlitz
> Sent: Thursday, May 24, 2007 1:19 AM
> To: Sean Hefty
> Cc: Roland Dreier (rdreier); general at lists.openfabrics.org
> Subject: [ofa-general] RE: two interfaces with ipoib
> 
> Sean Hefty wrote:
> > Steve Wise wrote:
> >> Guys, this reminds me of an issue we have with rnics and 
> regular nics 
> >> on the same physical network.  By default linux responds to arp 
> >> queries on all ports it receives the query on.  This leads 
> to very bad 
> >> results with you're trying to do offloaded connections.  When 
> >> resolving the address/route, the rdma client can end up 
> getting the 
> >> mac address of the dumb nic instead of the rnic.   I don't know if 
> >> route resolution in the ib cm has this issue, but it might 
> since they 
> >> use ipoib for some part of the resolution, no?
> > 
> > I think this could be the problem.  (And could have taken 
> me a long time 
> > to figure it if it is, so thanks!)
> > 
> >> 2) If you have a multi-homed host and the physical 
> ethernet networks are
> >> bridged, then you need to configure arp to only send replies on the
> >> interface with the target ip address:
> >>
> >>        sysctl -w net.ipv4.conf.all.arp_ignore=2
> 
> OK, Sean, sorry not to mention this to you, we have resolved 
> this with a 
> customer some time ago and I have communicated it to Mellanox 
> on Sonoma 
>   such that it will be added to the OFED 1.2 documentation.
> 
> Generally speaking, its a bad (somehow dead on arrival test 
> for a system 
> administrator) habit to have two IP (=L3) subnets sharing the same L2 
> (specifically broadcast) domain. In infiniband (IPoIB) it 
> means have two 
> IP subnets over the same Partition and in Ethernet is means 
> have two IP 
> subnets over the same VLAN.
> 
> I understand the default setting of arp_ignore = 0 is a religious 
> argument held once in a while at the netdev mailing list, if 
> people from 
> here want to try it, i am crossing my fingers for them, but again, it 
> has nothing special to do with ipoib.
> 
> Or.
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>