[openib-general] How to support IOMMUs for ipath driver

[Or Gerlitz]

Ralph Campbell wrote:
> Here is my thinking so far:
> 
> The driver is passed an LKEY/RKEY plus an address.
> For ib_get_dma_mr(), the address is currently from
> dma_map_single(), dma_map_page(), or dma_map_sg().
> With the ib_dma_*() routines, I can intercept these calls
> and return something instead of a bus or IOMMU address.
> I would like to return a kernel virtual address since that
> is the simplest and is what I ultimately need. This is
> trivial for dma_map_single() and trivial for low memory
> pages for dma_map_page().
> 
> I think I can safely just return error for architectures
> with high memory pages since the driver really only works
> on 64-bit systems (for a variety of reasons which I won't
> go into) and those systems don't have high memory.

Again (and please go and check me), pages you need to DMA (ie move over 
IB) need **not** be mapped into the kernel virtual address space and 
this happens **not** only under ia32 high-memory scheme, please see my 
other email for two examples (direct I/O etc)


> ib_sg_dma_address would return the page_address() of sg->page
> but wouldn't be able to rely on other fields which might be in
> the struct scatterlist.

your design seems to reply on three fields: page, offset and length, so

ib_sg_map_sg(scat) is kmap-ping whatever pages which are not mapped now 
into kvirt

ib_dma_unmap_sg(scat) is kunmap-ping those pages you were mapping before 
(you might need an aux data structure to keep which need kunmap)

ib_sg_dma_address(scat) is page_address(scat->page) + scat->offset

ib_sg_dma_len(scat) is scat->length

Or.