[ofw] RE: HPC server 2008 IPoIB disrupts compute node Windows Update service?

[Fab Tillier]

Hi Stan,

> -----Original Message-----
> From: Smith, Stan [mailto:stan.smith at intel.com]
> Sent: Monday, August 03, 2009 9:52 AM
>
> Hi Fab,
>   Do you have any theories on why/how IPoIB could disrupt Windows
> update service?
>
> We have an HPC Server 2008 cluster configured such that compute nodes,
> which are not directly connected to the enterprise network/Internet,
> use the Private Ethernet (NAT enabled) to communicate with the Head
> node which holds the enterprise/internet Ethernet interface.
> Once the compute nodes IE8 proxy is setup correctly, the compute node's
> windows update service works fine (downloads & install OK).
> Head-node Windows update service functions normally with IPoIB enable
> or disabled.
>
> After a WinOF installation, IPoIB enabled and static IPv4 address set
> on the compute node's IPoIB interface, the windows update service just
> hangs when windows update is run.
> It isn't until the IPoIB interface is disabled does the Windows update
> service resume normal operation?
>
> What is it about IPoIB and HPC compute nodes?

The OS isn't any different, so there shouldn't be anything special here.  However, if the Windows Update service tries to reach the internet using the IPoIB interface, it would likely fail since the IPoIB interface doesn't have NAT enabled for it.  I would have expected it to figure out the right interface to use though - are the subnet masks between the private Ethernet and IPoIB subnets distinct?

If you uninstall the HPC Pack from the compute node, does the problem persist or go away?

A netmon trace for the outbound traffic for both the IPoIB and non-IPoIB cases would likely shed a lot of light on the issue.

> We do not see the Windows update service problems when WinOF/IPoIB is
> installed on vanilla server 2008 or 2008/R2?

Do you also have the NAT in place in this scenario, on only the Ethernet network and not IPoIB?

-Fab