[ofa-general] Re: Demand paging for memory regions

[Caitlin Bestler]

Christoph Lameter wrote
> 
> > Merely mlocking pages deals with the end-to-end RDMA semantics.
> > What still needs to be addressed is how a fastpath interface
> > would dynamically pin and unpin. Yielding pins for short-term
> > suspensions (and flushing cached translations) deals with the
> > rest. Understanding the range of support that existing devices
> > could provide with software updates would be the next step if
> > you wanted to pursue this.
> 
> That is addressed on the VM level by the mmu_notifier which started
> this whole thread. The RDMA layers need to subscribe to this notifier
> and then do whatever the hardware requires to unpin and pin memory.
> I can only go as far as dealing with the VM layer. If you have any
> issues there I'd be glad to help.

There isn't much point in the RDMA layer subscribing to mmu
notifications
if the specific RDMA device will not be able to react appropriately when
the notification occurs. I don't see how you get around needing to know
which devices are capable of supporting page migration (via
suspend/resume
or other mechanisms) and which can only respond to a page migration by
aborting connections.