[Openib-windows] Windows DMA model

[Jan Bottorff]

>The SMI/GSI currently just call MmGetPhysicalAddress for MAD buffers,
and >this is due to change, either by allocating MADs from a common
buffer or by >doing DMA mappings when the MADs get posted to the QP
(likely the latter).  

So what your saying is currently, the IB driver does NOT assure correct
bus to processor physical address translation, and does NOT assure cache
coherency on systems that need software help for maintaining cache
coherency.

I aksed Microsoft about just calling MmGetPhysicalAddress for DMA and
they responded:



As an
optimization on the send side, all outgoing MADs could be sent inline
avoiding
the need for DMA, however this still requires the HCA driver to properly
map the
WQE ring, which gets to memory registration mappings.

Memory registrations don't do any DMA mappings at the moment, and that
is a
deficiency that I hope to correct when we move to the new verbs model (I
don't
know if you saw my RFC emails about that API or not).  This gets a bit
weird
though as the Windows DMA APIs take a flag that indicates which way the
DMA is
to occur, but don't support DMA in both directions through a single
mapping.
Also, driver verifier will completely break DMA for user-mode as it
forces
double buffering to check that DMA mappings are used properly.

There is some work that needs to happen to get MAD traffic to do proper
DMA
mappings, but upper level protocols already do the right thing.

For the time being, since we're running on platforms where the CPU and
bus
addresses are consistent, it hasn't been an issue.  I do want to see
this
change, though.

- Fab