[ofa-general] Re: [PATCH 1/11] IB/ipoib: high dma support

[Ralph Campbell]

On Tue, 2007-09-25 at 20:32 +0200, Michael S. Tsirkin wrote:
> > Quoting Ralph Campbell <ralph.campbell at qlogic.com>:
> > Subject: Re: [PATCH 1/11] IB/ipoib: high dma support
...
> > > I got the impression that all supported IB devices support dma-ing
> > > to/from memory > 4GB. Perhaps other vendors can comment.
> > 
> > The QLogic HCAs don't support DMA to or from the physical memory
> > for the verbs Lkey/Rkey memory regions. The whole reason I added
> > the ib_dma_*() functions was so to avoid ib_ipoib, etc. from
> > calling dma_*() directly and passing a physical address as the
> > offset in the posted work requests.
> > What happens instead, is that ib_dma_*() returns a kernel virtual
> > address which is passed in the work request and the driver copies
> > the data to/from the HW as needed.
> > So, in order to support HIGHMEM, I would need to change the
> > ipath_dma_*() functions to call kmap()/kunmap() for HIGHMEM pages.
> > I'm sure there would be all kinds of performance and coding issues
> > around doing this.
> 
> So, we need some kind of HIGHDMA capability flag?

Yes, A HIGHDMA capability flag would be useful.
I think that setting NETIF_F_HIGHDMA for ib_ipoib is a NOP for the
systems QLogic supports (i.e., only 64-bit kernels) but that could
change. Better to plan ahead.