[ofw] Loading drivers on LHS

[Fab Tillier]

There is a problem with kernel DLLs in that the dependencies when you
update devices is a nightmare to manage, since the driver doesn't show
up in any device stack.  The move away from a kernel DLL for complib and
IBAL was exactly to remove the problems with doing driver updates, where
there wasn't a good way to make sure that the IBAL kernel DLL was
unloaded before it needed to be updated.  For complib, it was also to
allow a mix of release and debug drivers to coexist.

 

Think of the case where you have a kernel driver that's a service,
something like SDP.  If IBAL was a DLL, to update the drivers you have
to know which services depend on it, disable all of them, disable all
the HCAs, and only then can you actually update the driver.  It's a
system management nightmare.

 

In any case, even splitting the driver up doesn't solve the problem for
user-mode clients like WSD that currently expect that the IBAL device is
available regardless of whether actual HW is present.

 

For kernel clients, the existing mechanism of querying for the IBAL
interface via IRP_MN_ QUERY_INTERFACE requests should not be changed.
This is the proper way for kernel drivers to get one another's
interfaces, and there is support for PnP notifications so that drivers
can be unloaded and updated.  Even querying the CI interface is fine.

 

Really, you want to stay away from kernel DLLs if you can.

 

So leave IBAL and the bus driver together, but make it a filter driver
for the HCA.  For user-mode devices, maybe we can also make the ibbus
driver a kernel service (managed through the service manager) so that it
gets loaded and creates the well known IBAL device object for user-mode
clients, without requiring a root enumerated device.  I don't think this
solves the issue of upgrading drivers - an admin will still need to stop
the service to update the drivers.  This might also require more code to
handle service manager requests.  It's also possible to make changes in
WSD so that the service isn't needed at all, so that only applications
that depend on it would require the service, and it would be disabled by
default.

 

-Fab

 

From: Yossi Leybovich [mailto:sleybo at mellanox.co.il] 
Sent: Thursday, May 10, 2007 5:33 AM
To: Fab Tillier; ofw at lists.openfabrics.org
Subject: RE: [ofw] Loading drivers on LHS

 

I think solution 2 is the way we need to go.

For the long term I would like to see  IBBUS.sys functionality split
into 2 modules (when we had ibbus.sys and ibal.sys)

 

1. The creation and management of the virtual devices should be
implemented as filter driver over the MTHCA ibbus.sys

2. all the VERBS functionality should implemented as kernel module
without any dependence to the HW.

 

 

Thanks

Yossi 

 

	
________________________________


	From: ofw-bounces at lists.openfabrics.org
[mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Fab Tillier
	Sent: Thursday, May 10, 2007 3:19 AM
	To: ofw at lists.openfabrics.org
	Subject: [ofw] Loading drivers on LHS

	Hi folks,

	 

	I tried loading the IB drivers on LHS (Windows Server code name
"Longhorn") and it doesn't work.  I tracked the problem down to the fact
that the driver packages are now staged to a special system folder
before being installed.  The IB co-installer uses the path of the
binaries to find the drivers for the "InfiniBand Fabric" device, but
since these didn't get staged with the HCA drivers, it fails and with it
installation of the HCA drivers fails.

	 

	I can't come up with a really good solution to this, so wanted
to start a discussion for how this should be addressed.

	 

	Solution 1: Don't fail the HCA driver installation if the
co-installer can't install the IB Bus device.

	While this allows the HCA drivers to install, it doesn't install
the IB Bus, which means no user-mode app will work (no IBAL device for
it to open), and no IPoIB or other child devices (no bus driver) will be
reported to the system.  A manual installation of the drivers for the IB
bus driver is required to get things really running, so there's no
longer an automatic installation path through the device manager.
Things can be scripted with devcon to work around this, though.

	 

	Solution 2: Make ibbus an upper filter for the HCA, include all
its information in the HCA's INF file.

	This solves the installation issue (and eliminates the
co-installer), and kernel clients will get created as needed, so it's a
pretty nice solution (it actually simplifies the PnP notifications
required of HCA drivers).  The problem is that it impacts user-mode
apps.  Currently, a user-mode client can load before any HCA is
available in the system, and use the IBAL PnP notifications to detect
arrival of an HCA.  If the IB bus driver is loaded as an upper filter,
it will not be loaded until an HCA is loaded, so user-mode applications
will no longer be able to rely on IBAL for CA PnP notifications.  This
would impact WSD for sure, though WSD depends on the IBAT device anyway
for address resolution, and this won't exist until the IB bus exists.
So WSD could be re-architected to delay-load IBAL until there are
addresses reported by IBAT.

	 

	Neither solution is all that great.  Any other thoughts and
ideas?

	 

	-Fab

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofw/attachments/20070510/48517be3/attachment.html>