[openib-general] problem with 2.6.19?

[Steve Wise]

I found the bug.  The ammasso driver was passing in the pointer
c2_dev->ibdev.dma_device to dma_map_single().  However the driver hadn't
yet registered with the ib core, so that field is zero.  The dma map
code uses bounce buffers if a map is done with a NULL device ptr passed
in...

The simple fix is to pass in the correct pointer, &c2dev->pcidev->dev.
But I guess I should still change this all to use
dma_alloc_coherent()...

Roland, Can we get this fix into 2.6.19?


Steve.




On Fri, 2006-10-27 at 11:15 -0700, Roland Dreier wrote:
>  > I must be misusing dma_map_single().  What I'm doing is allocating a
>  > verb message reply queue for the adapter to DMA verb replies into.  It
>  > never gets unmapped. I kmalloc() it, then map it.  I could use
>  > dma_alloc_coherent() or something, and maybe that's what I need to do?  
> 
> Yeah, if you want to leave something mapped and have the device DMA
> into it, and the CPU look into the buffer too, then you need
> consistent/coherent memory -- either pci_alloc_consistent() or
> dma_alloc_coherent().  The dma_ variant is slightly better because you
> can pass in a GFP_ mask rather than having the kernel pick GFP_ATOMIC
> for you.
> 
>  > You're saying I must unmap it before the data is valid (cuz of the
>  > bounce buffering).  If that's true, then how in sam hill does user mode
>  > RDMA work since the user's memory isn't unmapped before the user looks
>  > like memory that is the target of RDMA????  The uverbs code calls
>  > dma_map_sg() which is roughly the same as dma_map_single, eh?
> 
> It's a good point.  We're kind of counting on the IOMMU situation not
> being too wacky, and the device being able to DMA to arbitrary
> addresses.  So swiotlb won't work in this case actually -- but we
> assume any RDMA device can do 64-bit DMA so it doesn't hurt us in
> practice.  But yes, DMA to userspace is slightly risky and won't work
> everywhere.
> 
>  - R.