[Users] IPoIB not working on Windows 2008 r2 - need help
Orion Poplawski
orion at cora.nwra.com
Thu Jun 6 16:10:19 PDT 2013
On 06/06/2013 04:22 PM, Coulter, Susan K wrote:
>
> What does the arp cache look like on the host you are attempting to ping from ?
Interface: 192.168.2.200 --- 0x14
Internet Address Physical Address Type
192.168.2.255 ff-ff-ff-ff-ff-ff static
224.0.0.2 01-00-5e-00-00-02 static
224.0.0.22 01-00-5e-00-00-16 static
224.0.0.251 01-00-5e-00-00-fb static
224.0.0.252 01-00-5e-00-00-fc static
224.0.1.127 01-00-5e-00-01-7f static
239.2.11.71 01-00-5e-02-0b-47 static
239.192.83.80 01-00-5e-40-53-50 static
239.255.255.250 01-00-5e-7f-ff-fa static
239.255.255.253 01-00-5e-7f-ff-fd static
> You should be able to run tcpdump (windows equiv) on the IPoIB interfaces and see the arp/reply packets.
I see arp queries and others going out the interface, but nothing foreign
coming in.
> Are the multicast arp packets getting to this windows host?
I don't know. There look like some multicast addresses above, but I'm not to
familiar with this.
> The error messages from the osm log do seem to indicate a problem with someone trying to leave a multicast group, but I've not seen that exact message.
Yeah, my guess would be something messed up on the multicast side.
> Also - does native IB work - and just the IP emulation is broken?
>
Seems like it. ibdiagnet reports everything okay. I can use ibdiagpath to
get to some remote ports.
> On Jun 6, 2013, at 3:48 PM, Orion Poplawski <orion at cora.nwra.com> wrote:
>
>> On 06/06/2013 03:31 PM, Coulter, Susan K wrote:
>>>
>>> I am not a Windows person, but if ipconfig shows the IPoIB interface as up and running, then this seems like a routing problem on the node itself.
>>>
>>> The fact that the ICMP packets are being sent out a different NIC tells me the host has no route to the IPoIB space and is using a default route to send the packet out the 'default' interface.
>>>
>>
>> Yeah, that's what I thought too. But:
>>
>> ===========================================================================
>> Interface List
>> 20...00 05 ad 0c 5c ed ......Mellanox IPoIB Adapter
>> 17...00 30 48 cb 7c 91 ......Local Area Connection 2 - Virtual Network
>> 16...00 30 48 cb 7c 90 ......Local Area Connection - Virtual Network
>> 1...........................Software Loopback Interface 1
>> 11...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
>> 13...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
>> 14...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
>> 18...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3
>> ===========================================================================
>>
>> IPv4 Route Table
>> ===========================================================================
>> Active Routes:
>> Network Destination Netmask Gateway Interface Metric
>> 0.0.0.0 0.0.0.0 10.10.0.1 10.10.10.200 5
>> 10.10.0.0 255.255.0.0 On-link 10.10.10.200 261
>> 10.10.10.200 255.255.255.255 On-link 10.10.10.200 261
>> 10.10.255.255 255.255.255.255 On-link 10.10.10.200 261
>> 127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
>> 127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
>> 127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
>> 192.168.1.0 255.255.255.0 On-link 192.168.1.200 261
>> 192.168.1.200 255.255.255.255 On-link 192.168.1.200 261
>> 192.168.1.255 255.255.255.255 On-link 192.168.1.200 261
>> 192.168.2.0 255.255.255.0 On-link 192.168.2.200 261
>> 192.168.2.200 255.255.255.255 On-link 192.168.2.200 261
>> 192.168.2.255 255.255.255.255 On-link 192.168.2.200 261
>> 224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
>> 224.0.0.0 240.0.0.0 On-link 192.168.2.200 261
>> 224.0.0.0 240.0.0.0 On-link 10.10.10.200 261
>> 224.0.0.0 240.0.0.0 On-link 192.168.1.200 261
>> 255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
>> 255.255.255.255 255.255.255.255 On-link 192.168.2.200 261
>> 255.255.255.255 255.255.255.255 On-link 10.10.10.200 261
>> 255.255.255.255 255.255.255.255 On-link 192.168.1.200 261
>> ===========================================================================
>> Persistent Routes:
>> None
>>
>> Looks good to me.
>>
>> 192.168.2 is the IPoIB network/interface.
>> Windows IP Configuration
>>
>>
>> Ethernet adapter Local Area Connection 6:
>>
>> Connection-specific DNS Suffix . :
>> Link-local IPv6 Address . . . . . : fe80::817c:5b6e:6276:9ac6%20
>> IPv4 Address. . . . . . . . . . . : 192.168.2.200
>> Subnet Mask . . . . . . . . . . . : 255.255.255.0
>> Default Gateway . . . . . . . . . :
>>
>> Ethernet adapter Local Area Connection 5:
>>
>> Connection-specific DNS Suffix . :
>> Link-local IPv6 Address . . . . . : fe80::1463:384:4507:e704%17
>> IPv4 Address. . . . . . . . . . . : 192.168.1.200
>> Subnet Mask . . . . . . . . . . . : 255.255.255.0
>> Default Gateway . . . . . . . . . :
>>
>> Ethernet adapter Local Area Connection 4:
>>
>> Connection-specific DNS Suffix . : cora.nwra.com
>> Link-local IPv6 Address . . . . . : fe80::b989:f4ad:92c8:d660%16
>> IPv4 Address. . . . . . . . . . . : 10.10.10.200
>> Subnet Mask . . . . . . . . . . . : 255.255.0.0
>> Default Gateway . . . . . . . . . : 10.10.0.1
>>
>> Tunnel adapter isatap.{7FFCCABC-C5E1-4EFC-90A9-AF539A6FEB76}:
>>
>> Media State . . . . . . . . . . . : Media disconnected
>> Connection-specific DNS Suffix . :
>>
>> Tunnel adapter isatap.cora.nwra.com:
>>
>> Media State . . . . . . . . . . . : Media disconnected
>> Connection-specific DNS Suffix . : cora.nwra.com
>>
>> Tunnel adapter Teredo Tunneling Pseudo-Interface:
>>
>> Media State . . . . . . . . . . . : Media disconnected
>> Connection-specific DNS Suffix . :
>>
>> Tunnel adapter isatap.{4FC31CAB-4C54-4BBE-AC21-325440AEF9E5}:
>>
>> Media State . . . . . . . . . . . : Media disconnected
>> Connection-specific DNS Suffix . :
>>
>>
>> Hyper-V is running hence the isatap interfaces I believe. I have do not have a virtual interface connected to the IPoIB interface at the moment.
>>
>> It's so weird that it works for a few seconds and then goes away.
>>
>> --
>> Orion Poplawski
>> Technical Manager 303-415-9701 x222
>> NWRA, Boulder/CoRA Office FAX: 303-415-9702
>> 3380 Mitchell Lane orion at nwra.com
>> Boulder, CO 80301 http://www.nwra.com
>
> ====================================
>
> Susan Coulter
> HPC-3 Network/Infrastructure
> 505-667-8425
> Increase the Peace...
> An eye for an eye leaves the whole world blind
> ====================================
>
--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane orion at nwra.com
Boulder, CO 80301 http://www.nwra.com
More information about the Users
mailing list