[ofw] Mlx4 driver support

Wed Sep 2 14:37:15 PDT 2009

Leonid Keller wrote:
>> One reason is to allow install to finish.
>> When the driver goes at livefish mode it doesn't call create_pdo so
> windows doesn't know that a new driver has to be loaded.
> The last statement is not quite right. BUS driver creates all PDOs
> also in livefish mode (for the reason brought in the first statement).

If all PDOs are created then why can't Windows PNP find a device match for the mlx4_hca driver?

> One can even run vstat - which works over IBAL, HCA and BUS - and get
> the status of the card (while in livefish mode)

Since the Windows PNP is unable to match mlx4_hca to any PNP device, mlx4_hca.inf is not processed.
Unfortunately complib.dll is part of mlx4_hca.inf and without complib loaded into system32\, vstat fails as it can't find complib.dll.
The 'trick' would be to cd into %ProgramFiles%\WinOF\drivers\mlx4_hca\ and then run vstat to get the PSID.
Most WinOF users would not immediately grasp this trickery...
Bottom line is a causual WinOF user could be faced with a WinOF install which claimed success, no IPoIB instances, no vstat and no immediate way to figure out the PSID.....unless of course they also knew that flint could tell them....
Hence the PSID in event log error message suggestion.

>
>> a nice touch for your HCA customers would be to add the PSID to the
> event log error message ...
> There is no real need in that.
> Customer has gotten the main idea - he has to burn another FW.
> It can be done with TOOLS package.
> One of the tools is flint, which can print PSID.

I suspect that not everyone on the planet knows that flint will print a PSID...although more know now.  :-)

>
>> -----Original Message-----
>> From: Tzachi Dar
>> Sent: Wednesday, September 02, 2009 11:32 AM
>> To: Smith, Stan; Leonid Keller
>> Cc: ofw_list
>> Subject: RE: [ofw] Mlx4 driver support
>>
>> Please see bellow.
>>
>> Thanks
>> Tzachi
>>
>>> -----Original Message-----
>>> From: ofw-bounces at lists.openfabrics.org
>>> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Smith, Stan
>>> Sent: Tuesday, September 01, 2009 2:55 AM
>>> To: Leonid Keller; Tzachi Dar
>>> Cc: ofw_list
>>> Subject: RE: [ofw] Mlx4 driver support
>>>
>>> Leonid Keller wrote:
>>>> MLX4 HCA driver (mlx4_hca.sys) is HW-agnostic, it works with a
>>>> virtual HCA device. MLX4 BUS driver (mlx4_bus.sys) supports this
>>>> device - see mlx4_bus.inx:
>>>
>>> Hello,
>>>   Although the mlx4_hca.sys may be HW agnostic, it is not Firmware
>>> insensitive. The mlx4_hca.sys PNP load failure was caused by the
>>> mlx4_bus driver loading in a non-operation mode due to outdated
>>> firmware.
>>>
>>> BTW, a nice touch for your HCA customers would be to add the PSID to
>>> the event log error message so a customer would know which firmware
>>> image to download and flash!!
>>>
>>> system Event log entries in time order (#4 is latest).
>>>
>>> 4) mlx4_bus has started in non-operational mode.
>>>
>>> 3) QUERY_FW command failed with error -19.
>>>  The adapter card is non-functional.
>>>  Most likely a FW problem.
>>>  Please burn the last FW and restart the mlx4_bus driver.
>>>
>>> 2) HCA FW version 2.4.1002 is not supported. Use 2.5.0 or higher.
>>>
>>> 1)Driver Management has concluded the process to add Service
>>> mlx4_bus for  Device Instance ID
>>> PCI\VEN_15B3&DEV_673C&SUBSYS_673C15B3&REV_A0\6&1EC6E980&0&0020 0030
>>> with the following status: 0.
>>>
>>> What I'm really curious about is why the mlx4_bus driver returned
>>> load SUCCESS (as viewed from devmgmt.msc) when indeed the event log
>>> claims the device is non-operational?
>>
>> At the past burning new fw required the driver to load (the
>> tools were based on the hw driver).
>> As a result, we had to make sure that the driver still goes
>> up even the fw doesn't run.
>> The tools have changed and this behavior is not a must, but
>> it still remains. One reason is to allow install to finish.
>>
>>>
>>> Even more curious, is the Mellanox WinOF drivers load successfully
>>> while the open-source WinOF mlx4 HCA driver never loads as MSFT PNP
>>> based on the load status of mlx4_bus claims there is no match for
>>> mlx4_hca driver??
>>>
>> I'm not sure that I understand the question but what I can
>> tell is this:
>> 1) There is no real difference between the Mellanox WinOF and
>> the open source one. (so if one goes up, so should the other).
>> 2) When the driver goes at livefish mode it doesn't call
>> create_pdo so windows doesn't know that a new driver has to be
>> loaded.
>>
>>
>>
>>> What a tangled web....
>>>
>>> Stan.
>>>
>>>>
>>>> [MLX4BUS.DeviceSection.ntamd64]
>>>> ...
>>>> %MT26428.DeviceDesc%=MLX4BUS.DDInstall, PCI\VEN_15B3&DEV_673c
>>>>
>>>> I guess, you get the error while an unattended install on some
>>>> machine with the remains of some old installation.
>>>> Try to install these same driver manually from Device Manager,
>>>> while providing to the installer the right mlx4_bus.inx file ...
>>>>
>>>>> -----Original Message-----
>>>>> From: ofw-bounces at lists.openfabrics.org
>>>>> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Smith,
>>>>> Stan Sent: Friday, August 28, 2009 3:33 AM
>>>>> To: Tzachi Dar
>>>>> Cc: ofw_list
>>>>> Subject: [ofw] Mlx4 driver support
>>>>>
>>>>>
>>>>> Hello,
>>>>>   The SVN version of the mlx4 HCA driver does not appear to
>>>>> support the MT26428 rev A0 QDR with CX4 adapter?
>>>>> This is the 2nd time I've seen this failure on two different
>>>>> systems which have in common MT26428? It's interesting that the
>>>>> Mellanox Gold drivers install just fine....
>>>>>
>>>>> Suggestions on how to update the SVN mlx4 driver to support this
>>>>> hardware?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Stan.
>>>>>
>>>>>
>>>>>> From %windir%\INF\setupapi.dev.log after failed WinOF install.
>>>>>
>>>>>>>>  [DIF_SELECTBESTCOMPATDRV -
>>>>>>>>
>>>>>
>> PCI\VEN_15B3&DEV_673C&SUBSYS_673C15B3&REV_A0\6&1EC6E980&0&00200030]
>>>>>>>>  Section start 2009/08/27 15:49:23.024
>>>>>       cmd: rundll32.exe
>>>>> C:\Windows\system32\newdev.dll,pDiDeviceInstallAction
>>>>> \\.\pipe\PNP_Device_Install_Pipe_1.{c273630a-8f15-4e2f-9e56-46
>>>>> aa32c4c38e}
>>>>>
>>> "PCI\VEN_15B3&DEV_673C&SUBSYS_673C15B3&REV_A0\6&1ec6e980&0&00200030"
>>>>> !    dvi: Selecting driver failed(0xe0000228) !    dvi: Default
>>>>> installer: failed! !    dvi: Error 0xe0000228: There are no
>>>>> compatible drivers for this device.
>>>>> <<<  Section end 2009/08/27 15:49:23.024 <<<  [Exit status:
>>>>> FAILURE(0xe0000228)]
>>> _______________________________________________
>>>>> ofw mailing list ofw at lists.openfabrics.org
>>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>>>
>>> _______________________________________________
>>> ofw mailing list
>>> ofw at lists.openfabrics.org
>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw