[ofa-general] Re: [PATCH 08 of 11] anon-vma-rwsem

[Robin Holt]

On Tue, May 20, 2008 at 01:14:24PM +0200, Nick Piggin wrote:
> On Tue, May 20, 2008 at 06:05:29AM -0500, Robin Holt wrote:
> > On Tue, May 20, 2008 at 12:50:25PM +0200, Nick Piggin wrote:
> > > On Tue, May 20, 2008 at 05:01:11AM -0500, Robin Holt wrote:
> > > > On Tue, May 20, 2008 at 07:31:46AM +0200, Nick Piggin wrote:
> > > > > 
> > > > > Really? You can get the information through via a sleeping messaging API,
> > > > > but not a non-sleeping one? What is the difference from the hardware POV?
> > > > 
> > > > That was covered in the early very long discussion about 28 seconds.
> > > > The read timeout for the BTE is 28 seconds and it automatically retried
> > > > for certain failures.  In interrupt context, that is 56 seconds without
> > > > any subsequent interrupts of that or lower priority.
> > > 
> > > I thought you said it would be possible to get the required invalidate
> > > information without using the BTE. Couldn't you use XPMEM pages in
> > > the kernel to read the data out of, if nothing else?
> > 
> > I was wrong about that.  I thought it was safe to do an uncached write,
> > but it turns out any processor write is uncontained and the MCA that
> > surfaces would be fatal.  Likewise for the uncached read.
> 
> Oh, so the BTE transfer is purely for fault isolation. I was thinking
> you guys might have sufficient control of the hardware to be able to
> do it at the level of CPU memory operations, but if it is some
> limitation of ia64, then I guess that's a problem.
> 
> How do you do fault isolation of userspace XPMEM accesses?

The MCA handler can see the fault was either in userspace (processor
priviledge level I believe) or in the early kernel entry where it is
saving registers.  When it sees that condition, it kills the users
process.  While in kernel space, there is no equivalent of the saving
user state that forces the processor stall.