[openib-general] [RFC/BUG] libibverbs: DMA vs. CQ race

[Roland Dreier]

Are there other possible ordering problems involving user memory (not
in a CQ or QP)?  Something like a CPU on node A writing to memory on
node B and then posting a work request that makes the HCA DMA from
that memory on node B, and having the work request doorbell reach the
HCA before the write to node B actually happens, so the HCA DMAs the
old contents of node B's memory?

I guess the only feasible solution to the problem you're pointing out
is to have libmthca use some special mmap()-based allocator for queues
so that the kernel can give it memory that has the special
dma_map_consistent treatment.

Ugh.

 - R.