[ofa-general] ibv_reg_mr permissions?

[Troy Benjegerdes]

On Mar 6, 2007, at 3:16 AM, Guy German wrote:

>> I am running kernel 2.6.19.2, and I'm in the RDMA group, and can open
>> /dev/infiniband/uverbs0 as a user, but can't register memory as a  
>> user.
>
> try increasing your "max locked memory" limitation.
> you can try setting as root "ulimit -l unlimited" and switch user.
> if it works for you - set in /etc/security/limits.conf the needed  
> limitations for your user
>
> e.g.:
> user hard memlock 16384
> user soft memlock 16384
>
> Guy

Thanks!
Now, how does one determine what processes have registered memory  
regions, and how much memory is registered?

I have a testcase with NetPIPE and PVFS in which the process memory  
usage is about 100MB, but the free memory as reported by 'free' and  
'top' indicates about 3GB of memory is used. This is with the IBM  
ehca, which can apparently support registering a lot more memory than  
the mellanox PCI-X card I have in another power5 system. I don't see  
the extreme memory usage of 4GB with mthca, but it does appear that  
the free memory count drops more than what I would expect giving the  
process usage.