[ofa-general] Re: Demand paging for memory regions

[Christoph Lameter]

On Tue, 12 Feb 2008, Christian Bell wrote:

> I think there are very potential clients of the interface when an
> optimistic approach is used.  Part of the trick, however, has to do
> with being able to re-start transfers instead of buffering the data
> or making guarantees about delivery that could cause deadlock (as was
> alluded to earlier in this thread).  InfiniBand is constrained in
> this regard since it requires message-ordering between endpoints (or
> queue pairs).  One could argue that this is still possible with IB,
> at the cost of throwing more packets away when a referenced page is
> not in memory.  With this approach, the worse case demand paging
> scenario is met when the active working set of referenced pages is
> larger than the amount physical memory -- but HPC applications are
> already bound by this anyway.
> 
> You'll find that Quadrics has the most experience in this area and
> that their entire architecture is adapted to being optimistic about
> demand paging in RDMA transfers -- they've been maintaining a patchset
> to do this for years.

The notifier patchset that we are discussing here was mostly inspired by 
their work. 

There is no need to restart transfers that you have never started in the 
first place. The remote side would never start a transfer if the page 
reference has been torn down. In order to start the transfer a fault 
handler on the remote side would have to setup the association between the 
memory on both ends again.