[ofa-general] RE: Problem with using two interfaces with rdma-cm

[Sean Hefty]

Steve Wise wrote:
> Guys, this reminds me of an issue we have with rnics and regular nics on 
> the same physical network.  By default linux responds to arp queries on 
> all ports it receives the query on.  This leads to very bad results with 
> you're trying to do offloaded connections.  When resolving the 
> address/route, the rdma client can end up getting the mac address of the 
> dumb nic instead of the rnic.   I don't know if route resolution in the 
> ib cm has this issue, but it might since they use ipoib for some part of 
> the resolution, no?

I think this could be the problem.  (And could have taken me a long time 
to figure it if it is, so thanks!)

> 2) If you have a multi-homed host and the physical ethernet networks are
> bridged, then you need to configure arp to only send replies on the
> interface with the target ip address:
> 
>        sysctl -w net.ipv4.conf.all.arp_ignore=2

Sundeep, can you try this and see if it fixes the problem for you?

- Sean