[ofa-general] Re: InfiniBand card (mthca) in Linux

[Muli Ben-Yehuda]

On Mon, Jul 09, 2007 at 11:08:02AM +0200, Lukas Hejtmanek wrote:
> On Sun, Jul 08, 2007 at 10:03:16PM -0700, Roland Dreier wrote:
> > Is the memory given to a domU always physically contiguous?  If not,
> > what happens when a domU kernel does alloc_pages(GFP_KERNEL, 6) to try
> > and allocate 256 KB or something like that.  Let's assume that the
> > domU kernel has enough guest contiguous pages to satisfy the
> > allocation -- is there any guarantee that the pages are really
> > physically contiguous?
> 
> according to Xen-dev alloc_pages does *not* guarantee contiguous
> pages. They say that the pci_alloc_consistent should be used
> instead. The question is whether non-Xen kernel *usually* allocates
> contiguous pages and so far it has been working and whether it
> should be fixed in the mainline of the driver.
> 
> I do some tests (and also try to figure out how to change
> alloc_pages to pci_alloc_consistent) to verify contiguous pages.

You missed an important bit of Keir's response---it's perfectly fine
to use alloc_pages provided you then use the dma_map_single API, which
for Xen dom0 will take care of bounce-buffering to a
machine-contiguous buffer if necessary. I am not sure if the same
holds for a domU kernel.

Cheers,
Muli