[openib-general] Re: [PATCH][RFC][0/4] InfiniBand userspaceverbs implementation

[Bob Woodruff]

Roland wrote,
>I think you've missed the point: unless a process sets VM_DONTCOPY on
>its RDMA memory regions, then incorrect memory mappings may be used if
>the app does something as simple as calling system("ls").

> - R.

This is the exact problem that we saw with the Mellanox vapi driver.
It set VM_DONTCOPY and the result was that when someone did
a system("ls"), the call often caused a segv in the child. The issue seemed
to
be that someone had done a malloc of a buffer and then registered it,
which caused the entire page to be set to VM_DONTCOPY. Then someone
else (like the pthreads library) did a malloc that happened to reside
in the same page. When the user called system() which did a fork()/exec(),
the pthreads library would segv when trying to clean things up before
the exec(). 

We found that if we did not set VM_DONTCOPY that the child would no longer
segv, but in some instances, the Mellanox card seemed to be hosed after
the system call and would no longer transfer data. We never did understand
why.
We then found that if we set VM_DONTCOPY on only the register space
pages (doorbells and such), but not on the registered memory,
that it seemed to work OK. 

woody