[openib-general] Re: ibv_reg_mr/ memory locking

[Michael S. Tsirkin]

Quoting r. Roland Dreier <roland at topspin.com>:
> Subject: Re: ibv_reg_mr/ memory locking
> 
>     Michael> Are you aware of the fact that mlock wont always be
>     Michael> sufficient for this (notably COW)?  And, pages may be, or
>     Michael> become, COW, often without the user knowing.
> 
> Yes, in fact I think I was the one who originally debugged this.
> However I consider stuff like perfect fork() support a second-order
> issue.
> Most likely we will need a new VM operation that combines
> mlock and setting VM_DONTCOPY atomically, but I would hold off on that
> until we have a better understanding of the problems and solutions.

This is possible. But I think that VM_DONTCOPY affects whole pages.
So ibv_reg_mr would have to require page -aligned memory.
Not a huge problem in and of itself, but affects the API.


>     Michael> Some people do care about it
>     Michael> http://kerneltrap.org/node/4540 I dont know how
>     Michael> reasonable it is to make this unfixable by design.
> 
> This is all pretty encapsulated and can be ripped out later.
> Although
> if there are better ideas now, please bring them out...
> 
>  - R.

Okay.

I think its clear that if the doorbell itself was done from
the kernel context, we could lock the memory just before
the doorbell and unlock when that is done (and do a copy
if physical address moved).

So the administrator could limit the ability for raw memory
registrations to more priviledged users (we may need a new
rlimit for this, or something like that),
and let others get a bit higher latency but without the risk
of a DOS, and with lower memory requirements.

MST

-- 
I dont speak for Mellanox.