[ofa-general] Update on Roland's ummunotify kernel module

[Jeff Squyres]

Roland has pushed his new Linux "ummunotify" kernel upstream (i.e.,  
it's in his -next git branch):

http://git.kernel.org/?p=linux/kernel/git/roland/infiniband.git;a=commit;h=2fadea9acc19674c07ae7a9d90758f4b9b793940

It's not yet guaranteed that it will be accepted, but it looks good so  
far.  With some bug fixes from Pasha/Mellanox and Lenny+Mike/Voltaire,  
I think it's ready for wide-spread testing (I mailed some OMPI  
community members yesterday asking for specific testing).  I'm asking  
all MPI implementors to give the prototype code a whirl to shake out  
any remaining design bugs.  Others are welcome to review the design  
concepts and code as well; the more eyes, the better.  Bug fixes are  
easy later; design flaws are [much] better to be fixed now.

I describe the issue that we're fixing in my new MPI-themed blog:

    http://blogs.cisco.com/ciscotalk/performance/comments/better_linux_memory_tracking

The HG where this OMPI work is being done is here:

    http://bitbucket.org/jsquyres/ummunot/

You need to have a very recent Linux kernel (2.6.31+) and Roland's  
umunotify module installed/running.  Build the OMPI HG tree with the  
"--enable-mca-no-build=memory-ptmalloc2" to disable ptmalloc2 and  
enable the ummunotify stuff.

This hack-ish "disable ptmalloc2" step is only necessary while we're  
shaking out the design issues.  I'm halfway through merging the ummunot 
+ptmalloc2 code into a new opal/mca/memory component named "linux".   
This component will choose at run time whether to use ptmalloc2 or the  
ummunotify stuff (i.e., the --enable-mca-no-build... step won't be  
necessary when all is said and done; a default OMPI Linux build will  
do the Right Things).

Thanks.

-- 
Jeff Squyres
jsquyres at cisco.com