[Openib-windows] RE: Hibernation support

[Fab Tillier]

> From: Leonid Keller [mailto:leonid at mellanox.co.il]
> Sent: Thursday, January 12, 2006 1:47 AM
> 
> Hi Fab !
> 
> We have a client, that claims, that for getting
> WHQL certification we must provide support for
> hibernation and non-cache-coherent architectures
> (i.e. use DMA API in work with HCA cards).
> In that regard:
> 
> - could you comment on the claim (especially the
> second requirement looks to me strange) ?

The WHQL tests generally do public import tests.  The function
MmGetPhysicalAddress is often a restricted function, which would require ULPs to
do proper DMA mappings for all their I/O transfers.  There are a few areas where
this isn't done properly currently - the receive side of IPoIB, the information
units in the SRP miniport, and QP0/QP1 MAD traffic are the ones I know of off
the top of my head.

> - could you shortly describe possible ways of adding
> hibernation support with some TTM estimation ?

The first step in supporting hibernation would be proper power management
support in the HCA driver.  The current HCA driver will power the device down,
but not back up when the system resumes (I don't know what the new HCA driver
does).  The HCA driver must also keep track of user-allocated resources through
the hibernation process, and restore them to some state (potentially error) when
the system resumes operation. ULPs would need to handle these errors too and
properly recover.

All in all, I think it's a fairly significant change.  There are other changes
we will want to support to enable boot over IB, where the paging file,
hibernation file, and crashdump file might be on IB attached storage.  For these
to work, power management IRPs must be able to be processed at DISPATCH_LEVEL,
which they currently can't due to verb IRQL limitations.  There may well be
other issues we encounter as we try to support this.
 
> - If i'm not wrong, you once wanted to check with Microsoft
> the problems, which i have found in using of DMA API.
> Is there any progress with that ?

Calling GetScatterGatherList is the right way to handle DMA, as it will map all
or nothing, and never partial buffers.  For CQ and QP ring buffers, using
AllocateCommonBuffer is required since GetScatterGatherList might return the
addresses for Driver Verifier's buffers if DMA verification is enabled.

GetScatterGatherList will not work for user-mode ULPs like WSD if DMA
verification is enabled, and there's nothing we can do about it.  Kernel ULPs
that need to allocate common buffers for long term registrations.  For per-I/O
registrations, they should call GetScatterGatherList like any other ULP.

Personally, I would like to see these issues (hibernation, special file support,
DMA mappings) handled as part of the async verb work.  Basically, the current
stack was *ported* to Windows, rather than *designed* for it.  I'd like to come
up with a design for how handle the current issues, and then change the code to
implement the design.

For now, we need to focus on getting a first release done, and lack of
hibernation support will be a known issue.  Once we have the releases done, we
should start to work on adding the next level of functionality.  I would us to
be able to start on this work shortly, though any release-related issues will
have to take precedent.  At this point, I would guess we wouldn't have anything
concrete to solve these issues until the second half of the year.

- Fab