[openib-general] [PATCH] process locked in D state.

[Roland Dreier]

    Gleb> This is what happens: ibv_close_device() close cmd_fd and
    Gleb> then calls free_context().  free_context() calls munmap to
    Gleb> unmap doorbell registers.  In kernel sys_munmap gets
    Gleb> mm->mmap_sem semaphore and calls do_munmap.  do_munmap is
    Gleb> the last user of the file so it calls release method of the
    Gleb> file (ib_uverbs_close() in our case). ib_uverbs_close()
    Gleb> calls ib_dealloc_ucontext(). ib_dealloc_ucontext() notices
    Gleb> that there is unregistered memory on the file and calls
    Gleb> ib_umem_release(). And there we are trying to acquire
    Gleb> mm->mmap_sem on more time.

Thanks for the good debugging work.

    Gleb> In attached patch I use down_write_trylock() instead of
    Gleb> down_write() in ib_umem_release(). If semaphore is already
    Gleb> locked we will not update locked_vm statistics. This way
    Gleb> malicious user can only cause harm to itself.

I don't like this solution -- as you point out, down_write_trylock()
may fail if there is even momentary contention on the mmap_sem.  So
for example a different malicious user could poll on /proc/<pid>/maps
and cause our locked_vm to continue to grow.

How about if we use schedule_work() to defer the modification of
locked_vm?

 - R.