[openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation

[Andrew Morton]

Timur Tabi <timur.tabi at ammasso.com> wrote:
>
> Andrew Morton wrote:
> 
> > The way we expect get_user_pages() to be used is that the kernel will use
> > get_user_pages() once per application I/O request.
> > 
> > Are you saying that RDMA clients will semi-permanently own pages which were
> > pinned by get_user_pages()?  That those pages will be used for multiple
> > separate I/O operations?
> 
> Yes, absolutely!
> 
> The memory buffer is allocated by the process (usually just via malloc) and 
> registed/pinned by the driver.  It then stays pinned for the life of the process (typically).

ug.  What stops the memory from leaking if the process exits?

I hope this is a privileged operation?

> > If so, then that's a significant design departure and it would be good to
> > hear why it is necessary.
> 
> That's just how RMDA works.  Once the memory is pinned, if the app wants to send data to 
> another node, it does two things:
> 
> 1) Puts the data into its buffer
> 2) Sends a "work request" to the driver with (among other things) the offset and length of 
> the data.
> 
> This is a time-critical operation.  It must occurs as fast as possible, which means the 
> memory must have already been pinned.

It would be better to obtain this memory via a mmap() of some special
device node, so we can perform appropriate permission checking and clean
everything up on unclean application exit.