[ofa-general] Re: [PATCH 08 of 11] anon-vma-rwsem

[Nick Piggin]

On Fri, May 16, 2008 at 06:50:05AM -0500, Robin Holt wrote:
> On Fri, May 16, 2008 at 06:23:06AM -0500, Robin Holt wrote:
> > On Fri, May 16, 2008 at 01:52:03AM +0200, Nick Piggin wrote:
> > > On Thu, May 15, 2008 at 10:33:57AM -0700, Christoph Lameter wrote:
> > > > On Thu, 15 May 2008, Nick Piggin wrote:
> > > > 
> > > > > Oh, I get that confused because of the mixed up naming conventions
> > > > > there: unmap_page_range should actually be called zap_page_range. But
> > > > > at any rate, yes we can easily zap pagetables without holding mmap_sem.
> > > > 
> > > > How is that synchronized with code that walks the same pagetable. These 
> > > > walks may not hold mmap_sem either. I would expect that one could only 
> > > > remove a portion of the pagetable where we have some sort of guarantee 
> > > > that no accesses occur. So the removal of the vma prior ensures that?
> > >  
> > > I don't really understand the question. If you remove the pte and invalidate
> > > the TLBS on the remote image's process (importing the page), then it can
> > > of course try to refault the page in because it's vma is still there. But
> > > you catch that refault in your driver , which can prevent the page from
> > > being faulted back in.
> > 
> > I think Christoph's question has more to do with faults that are
> > in flight.  A recently requested fault could have just released the
> > last lock that was holding up the invalidate callout.  It would then
> > begin messaging back the response PFN which could still be in flight.
> > The invalidate callout would then fire and do the interrupt shoot-down
> > while that response was still active (essentially beating the inflight
> > response).  The invalidate would clear up nothing and then the response
> > would insert the PFN after it is no longer the correct PFN.
> 
> I just looked over XPMEM.  I think we could make this work.  We already
> have a list of active faults which is protected by a simple spinlock.
> I would need to nest this lock within another lock protected our PFN
> table (currently it is a mutex) and then the invalidate interrupt handler
> would need to mark the fault as invalid (which is also currently there).
> 
> I think my sticking points with the interrupt method remain at fault
> containment and timeout.  The inability of the ia64 processor to handle
> provide predictive failures for the read/write of memory on other
> partitions prevents us from being able to contain the failure.  I don't
> think we can get the information we would need to do the invalidate
> without introducing fault containment issues which has been a continous
> area of concern for our customers.

Really? You can get the information through via a sleeping messaging API,
but not a non-sleeping one? What is the difference from the hardware POV?