[Openib-windows] Windows DMA model

[Fab Tillier]

> From: Jan Bottorff [mailto:jbottorff at xsigo.com]
> Sent: Monday, January 23, 2006 3:48 PM
> 
> > In my understanding, a buffer, allocated by malloc, is
> > cachable, and we want to MAKE it unchacable both for CPU and
> > DMA. I guess, that in your understanding a Common Buffer is
> > already a SUCH one (i.e. - twice unchachable).
> 
> It may be impossible to do this. I believe at boot time memory is
> chopped up into regions and the processor MTRR registers are programmed
> with the memory properties (like caching or not) for each region. Most
> of memory is in a region that is fully cacheable/prefetchable. Memory
> reserved for common memory MAY be in a different region, although
> exactly what properties are set will depend on things like the I/O
> bridge architecture.
> 
> There also are bits in the page tables that indicate caching, and these
> MUST match the properties of the underlying memory (allocated from some
> MTRR defined region). If the MTRR and page tables don't have matching
> attributes, I believe the Intel processor manual describes processor
> behaivor as undefined.

Calls like MmMapLockedPagesSpecifyCache let you specify cache behavior, so I
would expect the OS to be able to handle runtime cache behavior specification.

> It's not just a matter of dreaming up any API that we want, it's a
> matter of defining API's which are implementable on current, recent
> past, and future processors based on underlying hardware constraints.

I totally agree.  From my queries into Microsoft, they are aware of the
deficiencies of the current DMA APIs for RDMA and similar hardware, and are
working on a solution.  I don't know more than that, however.

- Fab