[openib-general] routing weirdness

[Moni Shoua]

Michael S. Tsirkin wrote:
> Hi!
> I have a 2 port card with interfaces configured to different subnets.
> port 1 is down, port 2 is up.
> After some pinging, I see weird stuff in arp tables:
> does anyone have a clue why is linux trying to route
> packets to 12.4.3.87 trough ib0?
> 
> 
> sw086:~ # arp
> Address                  HWtype  HWaddress           Flags Mask            Iface
> mtlsv13.yok.mtl.com      ether   00:04:AC:A6:1F:B2   C                     eth0
> 10.4.0.211               ether   00:03:47:24:B3:A3   C                     eth0
> swlab37.yok.mtl.com      ether   00:02:B3:2B:2E:1B   C                     eth0
> mtls05.yok.mtl.com       ether   00:02:B3:2B:2B:9F   C                     eth0
> mtlsws02.yok.mtl.com     ether   00:02:B3:B0:3C:A6   C                     eth0
> sw086:~ # ping 11.4.3.87
> PING 11.4.3.87 (11.4.3.87) 56(84) bytes of data.
> 
> --- 11.4.3.87 ping statistics ---
> 2 packets transmitted, 0 received, 100% packet loss, time 1001ms
> 
> sw086:~ # ping 12.4.3.87
> PING 12.4.3.87 (12.4.3.87) 56(84) bytes of data.
> 
> --- 12.4.3.87 ping statistics ---
> 2 packets transmitted, 0 received, 100% packet loss, time 1003ms
> 
> sw086:~ # arp
> Address                  HWtype  HWaddress           Flags Mask            Iface
> 12.4.3.87                        (incomplete)                              ib0
> 11.4.3.87                        (incomplete)                              ib0
> mtlsv13.yok.mtl.com      ether   00:04:AC:A6:1F:B2   C                     eth0
> 10.4.0.211               ether   00:03:47:24:B3:A3   C                     eth0
> mtlsws09.yok.mtl.com     ether   00:E0:81:05:AF:F4   C                     eth0
> swlab37.yok.mtl.com      ether   00:02:B3:2B:2E:1B   C                     eth0
> mtls05.yok.mtl.com       ether   00:02:B3:2B:2B:9F   C                     eth0
> mtlsws02.yok.mtl.com     ether   00:02:B3:B0:3C:A6   C                     eth0
> sw086:~ # ifconfig ib0
> ib0       Link encap:UNSPEC  HWaddr 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00
>           inet addr:11.4.3.86  Bcast:11.4.255.255  Mask:255.255.0.0
>           UP BROADCAST MULTICAST  MTU:2044  Metric:1
>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:128
>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> 
> sw086:~ # ifconfig ib1
> ib1       Link encap:UNSPEC  HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00
>           inet addr:12.4.3.86  Bcast:12.4.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::202:c9ab:e987:fca2/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
>           RX packets:264 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:412 errors:0 dropped:290 overruns:0 carrier:0
>           collisions:0 txqueuelen:128
>           RX bytes:21868 (21.3 Kb)  TX bytes:33188 (32.4 Kb)
> 
> 

What do you see in the routing table? 
Which OS are you using?