[ofa-general] Re: InfiniBand card (mthca) in Linux

[Roland Dreier]

I don't know much about how Xen works, especially the PCI stuff in Xen
3.1.  So this may be a stupid idea, but anyway....

Is the memory given to a domU always physically contiguous?  If not,
what happens when a domU kernel does alloc_pages(GFP_KERNEL, 6) to try
and allocate 256 KB or something like that.  Let's assume that the
domU kernel has enough guest contiguous pages to satisfy the
allocation -- is there any guarantee that the pages are really
physically contiguous?

If not, what happens if the domU kernel does pci_map_sg() on an sglist
with >0 order pages in it that are not physically contigous?  The DMA
mapping API only allows one bus address to be returned for each page,
even if they are order >0 and hence more than 4 KB.  So if the pages
are guest contiguous but not physical host contiguous it seems we
could end up with the problem you see, where the domU mthca driver
tries to pass memory to the HCA but the HCA ends up writing to
different memory.

 - R.