[openib-general] Re: RDMA memory registration

[Caitlin Bestler]

On 5/3/05, David Addison <addy at quadrics.com> wrote:

> We believe the IOPROC patch is generic and powerful and would allow other
> RDMA NICs to solve the page registration problems in a different manner.
> For NICs which require page registration, new VM hooks can be used to avoid
> pages being unloaded whilst DMAs are active. Our latest cut of the IOPROC patch
> has such a hook.
> 

The key phrase here is "avoid pages being unloaded whilst DMAs are active".
Correct RDMA behavior requires preventing any loss of the content of those
pages in the period from the end of the DMA until the next completion is
reaped.

If the kernel were to start transferring the pages immediately after the DMA
completed, what would prevent the associated receive completion from being
generated before the migration was completed?

And if a migration is in progress, how is this feedback given to RDMA device
and when? Explicitly suspending a Memory Registration allows detection of
the problem while the disposition of the packet is still pending. Postponing
determination that the target memory is suspended until the actual DMA
transfer is attempted is problematic.