[ofa-general] RE: Problem with using two interfaces with rdma-cm

[Steve Wise]

Guys, this reminds me of an issue we have with rnics and regular nics on 
the same physical network.  By default linux responds to arp queries on 
all ports it receives the query on.  This leads to very bad results with 
you're trying to do offloaded connections.  When resolving the 
address/route, the rdma client can end up getting the mac address of the 
dumb nic instead of the rnic.   I don't know if route resolution in the 
ib cm has this issue, but it might since they use ipoib for some part of 
the resolution, no?

You might this (snipit from the cxgb3 release notes file to be included 
in -rc4):

2) If you have a multi-homed host and the physical ethernet networks are
bridged, then you need to configure arp to only send replies on the
interface with the target ip address:

        sysctl -w net.ipv4.conf.all.arp_ignore=2

Steve.

Sundeep Narravula wrote:
>> Odd - I will see if I can reproduce this.
>>
>> Are the HCAs sharing the same IB subnet?
>>     
>
> Yes. They are in the same IB subnet.
>
>   
>>> hmm.. I can try this but as an last resort. Ideally I would like to use
>>> just one listen cm_id binded to 0.0.0.0.
>>>       
>> I was thinking about binding on the active side, before calling connect.  But I
>> still want to look into this more.
>>     
>
> I can try this one out.
>
>   --Sundeep.
>
>   
>> - Sean
>>
>>     
>
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>