[openib-general] Re: [PATCH 5 of 13] ipath - use proper address translation routine

[Christoph Hellwig]

On Tue, May 02, 2006 at 07:24:18AM -0700, Roland Dreier wrote:
>     Christoph> Or stop doing the dma mapping in the IB upper level
>     Christoph> drivers.  I told you that we'll get broken hardware
>     Christoph> that doesn't want dma mapping in the upper level
>     Christoph> driver, and pathscale created exactly that :)
> 
> But see my earlier mail to Arjan about RDMA -- what address can a
> protocol (eg SRP initiator) put in a message that the other side will
> use to initiate a remote DMA operation?  It seems to me it has to be a
> bus address, and that means that the protocol has to do the DMA mapping.

Then we're back to the discussion on why RDMA is a fundamentally flawed
approach, but we already knew that.  The usual workaround is to only
allow RDMA operations to registered memory windows for which we can use
the normal dma operation.  There's also the *dac* pci dma operations that
can avoid iommu overhead if you support 64bit addressing.  But for all this
to work dma mapping fundamentally needs to be handled by the low level driver.