[ofa-general] Re: [patch 2/6] mmu_notifier: Callbacks to invalidate address ranges

[Andrea Arcangeli]

On Fri, Feb 29, 2008 at 01:34:34PM -0800, Christoph Lameter wrote:
> On Fri, 29 Feb 2008, Andrea Arcangeli wrote:
> 
> > On Fri, Feb 29, 2008 at 01:03:16PM -0800, Christoph Lameter wrote:
> > > That means we need both the anon_vma locks and the i_mmap_lock to become 
> > > semaphores. I think semaphores are better than mutexes. Rik and Lee saw 
> > > some performance improvements because list can be traversed in parallel 
> > > when the anon_vma lock is switched to be a rw lock.
> > 
> > The improvement was with a rw spinlock IIRC, so I don't see how it's
> > related to this.
> 
> AFAICT The rw semaphore fastpath is similar in performance to a rw 
> spinlock. 

read side is taken in the slow path.

write side is taken in the fast path.

pagefault is fast path, VM during swapping is slow path.

> > Perhaps the rwlock spinlock can be changed to a rw semaphore without
> > measurable overscheduling in the fast path. However theoretically
> 
> Overscheduling? You mean overhead?

The only possible overhead that a rw semaphore could ever generate vs
a rw lock is overscheduling.

> > speaking the rw_lock spinlock is more efficient than a rw semaphore in
> > case of a little contention during the page fault fast path because
> > the critical section is just a list_add so it'd be overkill to
> > schedule while waiting. That's why currently it's a spinlock (or rw
> > spinlock).
> 
> On the other hand a semaphore puts the process to sleep and may actually 
> improve performance because there is less time spend in a busy loop. 
> Other processes may do something useful and we stay off the contended 
> cacheline reducing traffic on the interconnect.

Yes, that's the positive side, the negative side is that you'll put
the task in uninterruptible sleep and call schedule() and require a
wakeup, because a list_add taking <1usec is running in the
other cpu. No other downside. But that's the only reason it's a
spinlock right now, infact there can't be any other reason.