[ofw] OFED for Windows on 2008R2 SP1

[Matthew Wallis]

Apparently we've found a solution for this. Of a sort.

The problem appears to be that SP1 comes with a driver for the Mellanox card. If we remove that driver, the machines actually stop booting. Nice.

Our current solution that obviously will require some more investigation is to install the official Mellanox drivers over the top of the Microsoft ones, then removing the Mellanox drivers. Once those are removed, the OFED drivers install without issue.

That's the long way round.

I can blacklist the driver in WSUS but I'm not sure I can stop SP1 from installing it.

Matt.

On 15/04/2011, at 9:52 AM, Matthew Wallis wrote:

> <ua-install.log><Screen shot 2011-04-15 at 9.32.59 AM.png>
> 
> Using the GUI, yes, IPoIB works fine. We've got two nodes running on GPFS from manual installs. Looks like it's a MT26428
> These are IBM branded cards with their own firmware on them. The Mellanox OEM and general drivers don't load at all on 
> these cards, we've had to push IBM to say they will support the OFED drivers.
> 
> [root at m1-m ~]# ibstat
> CA 'mlx4_0'
> 	CA type: MT26428
> 	Number of ports: 1
> 	Firmware version: 2.7.700
> 	Hardware version: b0
> 	Node GUID: 0x0002c903000d8d02
> 	System image GUID: 0x0002c903000d8d05
> 	Port 1:
> 		State: Active
> 		Physical state: LinkUp
> 		Rate: 40
> 		Base lid: 1
> 		LMC: 0
> 		SM lid: 1
> 		Capability mask: 0x0251086a
> 		Port GUID: 0x0002c903000d8d03
> 		Link layer: IB
> 
> 
> <Screen shot 2011-04-15 at 9.47.27 AM.png>
> 
> On 15/04/2011, at 1:49 AM, Smith, Stan wrote:
> 
>> Which ConnectX HCA and which firmware version?
>> 
>> Mellanox website has firmware burning tools if not latest firmware.
>> 
>> Please try the following install cmd:  start/wait msiexec /i OFED_2-3_win7_x64.msi /qn /log c:\ua-install.log
>> 
>> forward ua-install.log
>> 
>> Thanks,
>> 
>> Stan.
>> 
>>> -----Original Message-----
>>> From: Matthew Wallis [mailto:mattw at vpac.org]
>>> Sent: Wednesday, April 13, 2011 9:05 PM
>>> To: Smith, Stan
>>> Cc: ofw at lists.openfabrics.org
>>> Subject: Re: [ofw] OFED for Windows on 2008R2 SP1
>>> 
>>> 
>>> Just did the certificate based install using the msi, same results, only no prompting for certs this
>>> time
>> 
>> Progress.
>> 
>>> 
>>> z:\support\OFED> rem-cert-add.bat OFED OpenFabrics.cer
>>> Installing OpenFabrics.cer Cert on M1031
>>> 
>>> M1031 SUCCESS: OFA TrustedPublisher cert installed
>>> 
>>> z:\support\OFED>start/wait msiexec /i OFED_2-3_win7_x64.msi /qn
>>> 
>>> z:\support\OFED>
>>> 
>>> Had the usual flash message about the mlx_bus, but no IPoIB drivers, and vstat still complains about
>>> missing complib.dll
>> 
>> Since mlx4_bus does not appear to load correctly, then the mlx_hca driver will not be loaded, hence no complib.dll and vstat failure.
>> In the past, HCA firmware problems have caused this type of behavior; although no current idea as to why a GUI  install would work while the unattended install fails?
>> 
>> Speaking of a GUI install, does IPoIB configure correctly?
>> 
>>> 
>>> Matt.
>>> 
>>> On 14/04/2011, at 10:22 AM, Matthew Wallis wrote:
>>> 
>>>> We're doing an unattended install of Server 2008R2 HPC edition from a network share, so I
>>>> can either try to pre-seed that with the OFED drivers, or I can try and install the MSI via the HPC
>>>> admin console. I'll give them a shot. I do need to make sure I get IPoIB as we're using GPFS for
>>>> the filesystem.
>>>> 
>>> 
>>> --
>>> Matthew Wallis, HPC Systems Administrator
>>> Victorian Partnership for Advanced Computing.
>>> Ph: +61 3 9925 4452  Fax: +61 3 9925 4647
>> 
> 
> --
> Matthew Wallis, HPC Systems Administrator
> Victorian Partnership for Advanced Computing.
> Ph: +61 3 9925 4452  Fax: +61 3 9925 4647
> 
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw