RDMA memory registration (was: [openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspace verbs implementation)

[Roland Dreier]

Is there anything wrong with the following plan?

1) For memory registration, use get_user_pages() in the kernel.  Use
   locked_vm and RLIMIT_MEMLOCK to limit the amount of memory pinned
   by a given process.  One disadvantage of this is that the
   accounting will overestimate the amount of pinned memory if a
   process pins the same page twice, but this doesn't seem that bad to
   me -- it errs on the side of safety.

2) For fork() support:

   a) Extend mprotect() with PROT_DONTCOPY so processes can avoid
      copy-on-write problems.

   b) (maybe someday?) Add a VM_ALWAYSCOPY flag and extend mprotect()
      with PROT_ALWAYSCOPY so processes can mark pages to be
      pre-copied into child processes, to handle the case where only
      half a page is registered.

I believe this puts the code that must be trusted into the kernel and
gives userspace primitives that let apps handle the rest.

 - R.