[ofa-general] New proposal for memory management

[Jeff Squyres]

On Apr 30, 2009, at 1:30 AM, Aaron Fabbri (aafabbri) wrote:

> Have you considered changing the MPI API to require applications to  
> use MPI to
> allocate any/all buffers that may be used for network I/O?  That is,  
> instead of
> calling malloc() et al., call a new mpi_malloc() which allocates  
> from pre-
> registered memory.


Yes, MPI_ALLOC_MEM / MPI_FREE_MEM calls have been around for a long  
time (~10 years?).  Using them does avoid many of the problems that  
have been discussed.  Most (all?) MPI's either support ALLOC_MEM /  
FREE_MEM by registering at allocation time and unregistering at free  
time, or some variation of that.

But unfortunately, very few MPI apps use these calls; they use  
malloc() and friends instead.  Or they're written in Fortran, where  
such concepts are not easily mapped (don't underestimate how much  
Fortran MPI code runs on verbs!).  Indeed, in some layered scenarios,  
it's not easy to use these calls (e.g., if an MPI-enabled  
computational library may re-use user-provided buffers because they're  
so large, etc.).

-- 
Jeff Squyres
Cisco Systems