[Users] IPoIB not working on Windows 2008 r2 - need help
Coulter, Susan K
skc at lanl.gov
Thu Jun 6 15:22:43 PDT 2013
What does the arp cache look like on the host you are attempting to ping from ?
You should be able to run tcpdump (windows equiv) on the IPoIB interfaces and see the arp/reply packets.
Are the multicast arp packets getting to this windows host?
The error messages from the osm log do seem to indicate a problem with someone trying to leave a multicast group, but I've not seen that exact message.
Also - does native IB work - and just the IP emulation is broken?
On Jun 6, 2013, at 3:48 PM, Orion Poplawski <orion at cora.nwra.com> wrote:
> On 06/06/2013 03:31 PM, Coulter, Susan K wrote:
>>
>> I am not a Windows person, but if ipconfig shows the IPoIB interface as up and running, then this seems like a routing problem on the node itself.
>>
>> The fact that the ICMP packets are being sent out a different NIC tells me the host has no route to the IPoIB space and is using a default route to send the packet out the 'default' interface.
>>
>
> Yeah, that's what I thought too. But:
>
> ===========================================================================
> Interface List
> 20...00 05 ad 0c 5c ed ......Mellanox IPoIB Adapter
> 17...00 30 48 cb 7c 91 ......Local Area Connection 2 - Virtual Network
> 16...00 30 48 cb 7c 90 ......Local Area Connection - Virtual Network
> 1...........................Software Loopback Interface 1
> 11...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter
> 13...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #2
> 14...00 00 00 00 00 00 00 e0 Teredo Tunneling Pseudo-Interface
> 18...00 00 00 00 00 00 00 e0 Microsoft ISATAP Adapter #3
> ===========================================================================
>
> IPv4 Route Table
> ===========================================================================
> Active Routes:
> Network Destination Netmask Gateway Interface Metric
> 0.0.0.0 0.0.0.0 10.10.0.1 10.10.10.200 5
> 10.10.0.0 255.255.0.0 On-link 10.10.10.200 261
> 10.10.10.200 255.255.255.255 On-link 10.10.10.200 261
> 10.10.255.255 255.255.255.255 On-link 10.10.10.200 261
> 127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
> 127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
> 127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
> 192.168.1.0 255.255.255.0 On-link 192.168.1.200 261
> 192.168.1.200 255.255.255.255 On-link 192.168.1.200 261
> 192.168.1.255 255.255.255.255 On-link 192.168.1.200 261
> 192.168.2.0 255.255.255.0 On-link 192.168.2.200 261
> 192.168.2.200 255.255.255.255 On-link 192.168.2.200 261
> 192.168.2.255 255.255.255.255 On-link 192.168.2.200 261
> 224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
> 224.0.0.0 240.0.0.0 On-link 192.168.2.200 261
> 224.0.0.0 240.0.0.0 On-link 10.10.10.200 261
> 224.0.0.0 240.0.0.0 On-link 192.168.1.200 261
> 255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
> 255.255.255.255 255.255.255.255 On-link 192.168.2.200 261
> 255.255.255.255 255.255.255.255 On-link 10.10.10.200 261
> 255.255.255.255 255.255.255.255 On-link 192.168.1.200 261
> ===========================================================================
> Persistent Routes:
> None
>
> Looks good to me.
>
> 192.168.2 is the IPoIB network/interface.
> Windows IP Configuration
>
>
> Ethernet adapter Local Area Connection 6:
>
> Connection-specific DNS Suffix . :
> Link-local IPv6 Address . . . . . : fe80::817c:5b6e:6276:9ac6%20
> IPv4 Address. . . . . . . . . . . : 192.168.2.200
> Subnet Mask . . . . . . . . . . . : 255.255.255.0
> Default Gateway . . . . . . . . . :
>
> Ethernet adapter Local Area Connection 5:
>
> Connection-specific DNS Suffix . :
> Link-local IPv6 Address . . . . . : fe80::1463:384:4507:e704%17
> IPv4 Address. . . . . . . . . . . : 192.168.1.200
> Subnet Mask . . . . . . . . . . . : 255.255.255.0
> Default Gateway . . . . . . . . . :
>
> Ethernet adapter Local Area Connection 4:
>
> Connection-specific DNS Suffix . : cora.nwra.com
> Link-local IPv6 Address . . . . . : fe80::b989:f4ad:92c8:d660%16
> IPv4 Address. . . . . . . . . . . : 10.10.10.200
> Subnet Mask . . . . . . . . . . . : 255.255.0.0
> Default Gateway . . . . . . . . . : 10.10.0.1
>
> Tunnel adapter isatap.{7FFCCABC-C5E1-4EFC-90A9-AF539A6FEB76}:
>
> Media State . . . . . . . . . . . : Media disconnected
> Connection-specific DNS Suffix . :
>
> Tunnel adapter isatap.cora.nwra.com:
>
> Media State . . . . . . . . . . . : Media disconnected
> Connection-specific DNS Suffix . : cora.nwra.com
>
> Tunnel adapter Teredo Tunneling Pseudo-Interface:
>
> Media State . . . . . . . . . . . : Media disconnected
> Connection-specific DNS Suffix . :
>
> Tunnel adapter isatap.{4FC31CAB-4C54-4BBE-AC21-325440AEF9E5}:
>
> Media State . . . . . . . . . . . : Media disconnected
> Connection-specific DNS Suffix . :
>
>
> Hyper-V is running hence the isatap interfaces I believe. I have do not have a virtual interface connected to the IPoIB interface at the moment.
>
> It's so weird that it works for a few seconds and then goes away.
>
> --
> Orion Poplawski
> Technical Manager 303-415-9701 x222
> NWRA, Boulder/CoRA Office FAX: 303-415-9702
> 3380 Mitchell Lane orion at nwra.com
> Boulder, CO 80301 http://www.nwra.com
====================================
Susan Coulter
HPC-3 Network/Infrastructure
505-667-8425
Increase the Peace...
An eye for an eye leaves the whole world blind
====================================
More information about the Users
mailing list