[ofa-general] Re: Demand paging for memory regions

[Christoph Lameter]

On Wed, 13 Feb 2008, Jason Gunthorpe wrote:

> Unfortunately it really has little to do with the drivers - changes,
> for instance, need to be made to support this in the user space MPI
> libraries. The RDMA ops do not pass through the kernel, userspace
> talks directly to the hardware which complicates building any sort of
> abstraction.

Ok so the notifiers have to be handed over to the user space library that 
has the function of the device driver here...

> That is where I think you run into trouble, if you ask the MPI people
> to add code to their critical path to support swapping they probably
> will not be too interested. At a minimum to support your idea you need
> to check on every RDMA if the remote page is mapped... Plus the
> overheads Christian was talking about in the OOB channel(s).

You only need to check if a handle has been receiving invalidates. If not 
then you can just go ahead as now. You can use the notifier to take down 
the whole region if any reclaim occur against it (probably best and 
simples to implement approach). Then you mark the handle so that the 
mapping is reestablished before the next operation.