[ofw] Mlx4 driver support
Leonid Keller
leonid at mellanox.co.il
Thu Sep 3 02:53:02 PDT 2009
I saw below "%ProgramFiles%\WinOF\drivers\mlx4_hca\" and suddenly got
that we misunderstood you from the very beginning (at least - me).
We install drivers in MLNX_WinOF\driver directory, so for us 'mlx4_hca'
is a name of the concrete virtual driver mlx4_hca.sys with hw_id
"MLX4\ConnectX_Hca".
And you sure meant the bus driver (mlx4_bus.sys).
All PDOs are created in case when mlx4_bus has *succeeded* to start,
also in livefish mode.
If OS failed to find it, no other driver will be loaded.
Now, I see some inconsistency in your reports.
If mlx4_bus was not found and loaded, then you are not expected to see
any messages in Event Log.
If you do see them, like "mlx4_bus has started in non-operational mode",
it means that mlx4_bus *has started* and it should have built all the
stack - mlx4_hca - ibbus - winverbs - winmad - ipoib and vstat should
also work.
We never saw situation when mlx4bus was loaded and other drivers - not.
Have you tried to install drivers manually as I suggested ?
With regard to PSID:
- I remember times when we didn't support PSID at all;
- I'm not sure that all users know what it is and what's it role
in the burning of FW;
- I'm also not sure VSTAT prints PSID while in livefish mode.
And if it can't so do the driver;
- the Event Log message states quite clearly:
"Please burn the last FW and restart the mlx4_bus
driver"
So the customer *has to* achieve the TOOLS package and study
how to use it.
> -----Original Message-----
> From: Smith, Stan [mailto:stan.smith at intel.com]
> Sent: Thursday, September 03, 2009 12:37 AM
> To: Leonid Keller; Tzachi Dar
> Cc: ofw_list
> Subject: RE: [ofw] Mlx4 driver support
>
> Leonid Keller wrote:
> >> One reason is to allow install to finish.
> >> When the driver goes at livefish mode it doesn't call create_pdo so
> > windows doesn't know that a new driver has to be loaded.
> > The last statement is not quite right. BUS driver creates all PDOs
> > also in livefish mode (for the reason brought in the first
> statement).
>
> If all PDOs are created then why can't Windows PNP find a
> device match for the mlx4_hca driver?
>
> > One can even run vstat - which works over IBAL, HCA and BUS
> - and get
> > the status of the card (while in livefish mode)
>
> Since the Windows PNP is unable to match mlx4_hca to any PNP
> device, mlx4_hca.inf is not processed.
> Unfortunately complib.dll is part of mlx4_hca.inf and without
> complib loaded into system32\, vstat fails as it can't find
> complib.dll.
> The 'trick' would be to cd into
> %ProgramFiles%\WinOF\drivers\mlx4_hca\ and then run vstat to
> get the PSID.
> Most WinOF users would not immediately grasp this trickery...
> Bottom line is a causual WinOF user could be faced with a
> WinOF install which claimed success, no IPoIB instances, no
> vstat and no immediate way to figure out the PSID.....unless
> of course they also knew that flint could tell them....
> Hence the PSID in event log error message suggestion.
>
> >
> >> a nice touch for your HCA customers would be to add the PSID to the
> > event log error message ...
> > There is no real need in that.
> > Customer has gotten the main idea - he has to burn another FW.
> > It can be done with TOOLS package.
> > One of the tools is flint, which can print PSID.
>
> I suspect that not everyone on the planet knows that flint
> will print a PSID...although more know now. :-)
>
> >
> >> -----Original Message-----
> >> From: Tzachi Dar
> >> Sent: Wednesday, September 02, 2009 11:32 AM
> >> To: Smith, Stan; Leonid Keller
> >> Cc: ofw_list
> >> Subject: RE: [ofw] Mlx4 driver support
> >>
> >> Please see bellow.
> >>
> >> Thanks
> >> Tzachi
> >>
> >>> -----Original Message-----
> >>> From: ofw-bounces at lists.openfabrics.org
> >>> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of
> Smith, Stan
> >>> Sent: Tuesday, September 01, 2009 2:55 AM
> >>> To: Leonid Keller; Tzachi Dar
> >>> Cc: ofw_list
> >>> Subject: RE: [ofw] Mlx4 driver support
> >>>
> >>> Leonid Keller wrote:
> >>>> MLX4 HCA driver (mlx4_hca.sys) is HW-agnostic, it works with a
> >>>> virtual HCA device. MLX4 BUS driver (mlx4_bus.sys) supports this
> >>>> device - see mlx4_bus.inx:
> >>>
> >>> Hello,
> >>> Although the mlx4_hca.sys may be HW agnostic, it is not
> Firmware
> >>> insensitive. The mlx4_hca.sys PNP load failure was caused by the
> >>> mlx4_bus driver loading in a non-operation mode due to outdated
> >>> firmware.
> >>>
> >>> BTW, a nice touch for your HCA customers would be to add
> the PSID to
> >>> the event log error message so a customer would know
> which firmware
> >>> image to download and flash!!
> >>>
> >>> system Event log entries in time order (#4 is latest).
> >>>
> >>> 4) mlx4_bus has started in non-operational mode.
> >>>
> >>> 3) QUERY_FW command failed with error -19.
> >>> The adapter card is non-functional.
> >>> Most likely a FW problem.
> >>> Please burn the last FW and restart the mlx4_bus driver.
> >>>
> >>> 2) HCA FW version 2.4.1002 is not supported. Use 2.5.0 or higher.
> >>>
> >>> 1)Driver Management has concluded the process to add Service
> >>> mlx4_bus for Device Instance ID
> >>>
> PCI\VEN_15B3&DEV_673C&SUBSYS_673C15B3&REV_A0\6&1EC6E980&0&0020 0030
> >>> with the following status: 0.
> >>>
> >>> What I'm really curious about is why the mlx4_bus driver returned
> >>> load SUCCESS (as viewed from devmgmt.msc) when indeed the
> event log
> >>> claims the device is non-operational?
> >>
> >> At the past burning new fw required the driver to load (the tools
> >> were based on the hw driver).
> >> As a result, we had to make sure that the driver still
> goes up even
> >> the fw doesn't run.
> >> The tools have changed and this behavior is not a must,
> but it still
> >> remains. One reason is to allow install to finish.
> >>
> >>>
> >>> Even more curious, is the Mellanox WinOF drivers load
> successfully
> >>> while the open-source WinOF mlx4 HCA driver never loads
> as MSFT PNP
> >>> based on the load status of mlx4_bus claims there is no match for
> >>> mlx4_hca driver??
> >>>
> >> I'm not sure that I understand the question but what I can tell is
> >> this:
> >> 1) There is no real difference between the Mellanox WinOF and the
> >> open source one. (so if one goes up, so should the other).
> >> 2) When the driver goes at livefish mode it doesn't call
> create_pdo
> >> so windows doesn't know that a new driver has to be loaded.
> >>
> >>
> >>
> >>> What a tangled web....
> >>>
> >>> Stan.
> >>>
> >>>>
> >>>> [MLX4BUS.DeviceSection.ntamd64]
> >>>> ...
> >>>> %MT26428.DeviceDesc%=MLX4BUS.DDInstall, PCI\VEN_15B3&DEV_673c
> >>>>
> >>>> I guess, you get the error while an unattended install on some
> >>>> machine with the remains of some old installation.
> >>>> Try to install these same driver manually from Device Manager,
> >>>> while providing to the installer the right mlx4_bus.inx file ...
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: ofw-bounces at lists.openfabrics.org
> >>>>> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Smith,
> >>>>> Stan Sent: Friday, August 28, 2009 3:33 AM
> >>>>> To: Tzachi Dar
> >>>>> Cc: ofw_list
> >>>>> Subject: [ofw] Mlx4 driver support
> >>>>>
> >>>>>
> >>>>> Hello,
> >>>>> The SVN version of the mlx4 HCA driver does not appear to
> >>>>> support the MT26428 rev A0 QDR with CX4 adapter?
> >>>>> This is the 2nd time I've seen this failure on two different
> >>>>> systems which have in common MT26428? It's interesting that the
> >>>>> Mellanox Gold drivers install just fine....
> >>>>>
> >>>>> Suggestions on how to update the SVN mlx4 driver to
> support this
> >>>>> hardware?
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> Stan.
> >>>>>
> >>>>>
> >>>>>> From %windir%\INF\setupapi.dev.log after failed WinOF install.
> >>>>>
> >>>>>>>> [DIF_SELECTBESTCOMPATDRV -
> >>>>>>>>
> >>>>>
> >> PCI\VEN_15B3&DEV_673C&SUBSYS_673C15B3&REV_A0\6&1EC6E980&0&00200030]
> >>>>>>>> Section start 2009/08/27 15:49:23.024
> >>>>> cmd: rundll32.exe
> >>>>> C:\Windows\system32\newdev.dll,pDiDeviceInstallAction
> >>>>> \\.\pipe\PNP_Device_Install_Pipe_1.{c273630a-8f15-4e2f-9e56-46
> >>>>> aa32c4c38e}
> >>>>>
> >>>
> "PCI\VEN_15B3&DEV_673C&SUBSYS_673C15B3&REV_A0\6&1ec6e980&0&00200030"
> >>>>> ! dvi: Selecting driver failed(0xe0000228) ! dvi: Default
> >>>>> installer: failed! ! dvi: Error 0xe0000228: There are no
> >>>>> compatible drivers for this device.
> >>>>> <<< Section end 2009/08/27 15:49:23.024 <<< [Exit status:
> >>>>> FAILURE(0xe0000228)]
> >>> _______________________________________________
> >>>>> ofw mailing list ofw at lists.openfabrics.org
> >>>>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
> >>>
> >>> _______________________________________________
> >>> ofw mailing list
> >>> ofw at lists.openfabrics.org
> >>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>
>
More information about the ofw
mailing list