[ofa-general] Memory registration redux

[Caitlin Bestler]

On Thu, May 7, 2009 at 3:48 PM, Jason Gunthorpe
<jgunthorpe at obsidianresearch.com> wrote:
>
> Right, I was only thinking of a new driver call that was along the
> lines of update_mr_pages() that just updates the HCA's mapping with
> new page table entires atomically. It really would be device
> specific. If there is no call available then unregister/register +
> printk log is a fair generic implementation.
>
> To be clear, what I'm thinking is that this would only be invoked if

Both the IBTA and RDMAC verbs were defined so that the meaning of
L-Key/R-Key/STag + Address could not
instantly change from "X" to "Y", only from "X" to NULL and then NULL to "Y".

There are a lot of good reasons for this, especially for R-Keys or
remotely accessible STags. It ensures that
all operations that started when the translation was "X" are completed
before any that will use the "Y" translation
can commence. That is not something we want to accidentally undermine.

There really isn't a reason why this rule needed to apply to entire
Memory Regions. So I don't see a problem
with allowing an update_mr_pages() verb that changes a portion of an
MR map, perhaps by optimal machine
specific hooks when available, without requiring the entire MR be
specified. But it must preserve the guarantee
that all operations initiated with translation "X" are completed
before any operations for translation "Y" can be initiated.

Preserving this guarantee should not be a problem for the free() then
reallocate scenarios that have been discussed.