[openib-general] Question about pinning memory

[Gleb Natapov]

On Mon, Jul 25, 2005 at 07:16:48AM -0700, Roland Dreier wrote:
>     Pete> I did some thinking about this issue a while back and came
>     Pete> up with a cooperative kernel/user implementation to track
>     Pete> linux VM activity using existing vm_area_struct->vm_ops
>     Pete> function hooks (i.e. no kernel patch required).  The MPI
>     Pete> library essentially makes a system call before reusing a
>     Pete> cached memory registration to verify it is still valid, and
>     Pete> the kernel module keeps track of what happens to cached
>     Pete> mappings as the VM system is exercised via sbrk, mmap, fork,
>     Pete> etc.  It works for any sort of memory activity, including
>     Pete> arbitrary mmap() of memory or files, since it plugs in at
>     Pete> the basic VM interface level.
> 
> This seems like the right approach to me.  I didn't like the patches
> posted by Quadrics a while ago, since they added hooks to all of
> low-level page table hot paths.  Working at the vma level seems much
> saner.
> 
> How much of a performance hit is this system call?  It seems that you
> need to do the call before every single work request is posted, which
> could cause latency to go way up.
I think the better approach would be to send signal to process whenever
vma list changes and let application verify whether cache entries
should be freed.

What do you think?

> 
>     Pete> I can distribute the code if anyone is curious, but it needs
>     Pete> some work to become production quality.
> 
> Yes, please post the code soon.
> 

--
			Gleb.