[openib-general] Re: Question about pinning memory

[Michael S. Tsirkin]

Quoting r. Pete Wyckoff <pw at osc.edu>:
> Subject: Re: Question about pinning memory
> 
> jsquyres at open-mpi.org wrote on Fri, 22 Jul 2005 19:04 -0400:
> > Otherwise, the scenario in question #3 is a real problem.  There are a 
> > few possibilities for fixing it, but all are problematic (override 
> > sbrk() via including ptmalloc2 in the distribution, using LD_PRELOAD to 
> > override sbrk(), etc.).  Any other suggestions would be welcome...
> 
> I did some thinking about this issue a while back and came up with a
> cooperative kernel/user implementation to track linux VM activity using
> existing vm_area_struct->vm_ops function hooks (i.e. no kernel patch
> required).  The MPI library essentially makes a system call before
> reusing a cached memory registration to verify it is still valid, and
> the kernel module keeps track of what happens to cached mappings as the
> VM system is exercised via sbrk, mmap, fork, etc.  It works for any sort
> of memory activity, including arbitrary mmap() of memory or files, since
> it plugs in at the basic VM interface level.

Since this seems to imply that a system call is performed on each send/receive
operation, why not just use SDP, or a similiar kernel level protocol?

-- 
MST