[openib-general] problem with 2.6.19?

[Muli Ben-Yehuda]

On Fri, Oct 27, 2006 at 11:15:40AM -0700, Roland Dreier wrote:
>  > I must be misusing dma_map_single().  What I'm doing is allocating a
>  > verb message reply queue for the adapter to DMA verb replies into.  It
>  > never gets unmapped. I kmalloc() it, then map it.  I could use
>  > dma_alloc_coherent() or something, and maybe that's what I need to do?  
> 
> Yeah, if you want to leave something mapped and have the device DMA
> into it, and the CPU look into the buffer too, then you need
> consistent/coherent memory -- either pci_alloc_consistent() or
> dma_alloc_coherent().  The dma_ variant is slightly better because you
> can pass in a GFP_ mask rather than having the kernel pick GFP_ATOMIC
> for you.

You can also use the sync_{single|sg}_for_{device|cpu} calls to manually sync
the buffers (this will cause a memcpy for swiotlb).

>  > You're saying I must unmap it before the data is valid (cuz of the
>  > bounce buffering).

No, sync_xxx should work as well to make the data valid.

>  > If that's true, then how in sam hill does user mode
>  > RDMA work since the user's memory isn't unmapped before the user looks
>  > like memory that is the target of RDMA????  The uverbs code calls
>  > dma_map_sg() which is roughly the same as dma_map_single, eh?
> 
> It's a good point.  We're kind of counting on the IOMMU situation not
> being too wacky,

Could you elaborate, what are the "requirements" for this to work with
an arbitrary IOMMU (e.g., Calgary) than requires a mapping to exist
before a DMA operation can go through?

Cheers,
Muli