[ofa-general] Re: [PATCH 08 of 11] anon-vma-rwsem

[Robin Holt]

On Fri, May 16, 2008 at 06:23:06AM -0500, Robin Holt wrote:
> On Fri, May 16, 2008 at 01:52:03AM +0200, Nick Piggin wrote:
> > On Thu, May 15, 2008 at 10:33:57AM -0700, Christoph Lameter wrote:
> > > On Thu, 15 May 2008, Nick Piggin wrote:
> > > 
> > > > Oh, I get that confused because of the mixed up naming conventions
> > > > there: unmap_page_range should actually be called zap_page_range. But
> > > > at any rate, yes we can easily zap pagetables without holding mmap_sem.
> > > 
> > > How is that synchronized with code that walks the same pagetable. These 
> > > walks may not hold mmap_sem either. I would expect that one could only 
> > > remove a portion of the pagetable where we have some sort of guarantee 
> > > that no accesses occur. So the removal of the vma prior ensures that?
> >  
> > I don't really understand the question. If you remove the pte and invalidate
> > the TLBS on the remote image's process (importing the page), then it can
> > of course try to refault the page in because it's vma is still there. But
> > you catch that refault in your driver , which can prevent the page from
> > being faulted back in.
> 
> I think Christoph's question has more to do with faults that are
> in flight.  A recently requested fault could have just released the
> last lock that was holding up the invalidate callout.  It would then
> begin messaging back the response PFN which could still be in flight.
> The invalidate callout would then fire and do the interrupt shoot-down
> while that response was still active (essentially beating the inflight
> response).  The invalidate would clear up nothing and then the response
> would insert the PFN after it is no longer the correct PFN.

I just looked over XPMEM.  I think we could make this work.  We already
have a list of active faults which is protected by a simple spinlock.
I would need to nest this lock within another lock protected our PFN
table (currently it is a mutex) and then the invalidate interrupt handler
would need to mark the fault as invalid (which is also currently there).

I think my sticking points with the interrupt method remain at fault
containment and timeout.  The inability of the ia64 processor to handle
provide predictive failures for the read/write of memory on other
partitions prevents us from being able to contain the failure.  I don't
think we can get the information we would need to do the invalidate
without introducing fault containment issues which has been a continous
area of concern for our customers.


Thanks,
Robin