[ofa-general] Re: Re: [PATCH RFC] sharing userspace IB objects

[Michael S. Tsirkin]

> Quoting Or Gerlitz <ogerlitz at voltaire.com>:
> Subject: Re: Re: [PATCH RFC] sharing userspace IB objects
> 
> Michael S. Tsirkin wrote:
> >>Quoting Roland Dreier <rdreier at cisco.com>:
> >>Subject: Re: [PATCH RFC] sharing userspace IB objects
> 
> >>   Is there really a strong use case for making every type of object
> >>   shareable?  Can we handle the SRC stuff without going to this
> >>   extreme of complexity?
> 
> >This is not directly related to SRC: this is an effort
> >to make it possible to share QPs, CQ etc across processes
> >in the same way as they can be currently shared across threads.
> >So assuming that we want multiple processes to post to
> >the same QP, how can we support this?
> 
> Indeed, lets zoom out a little and define the high level scope and 
> design here, such that people can comment.

What I want to do is make it possible to share libibverbs objects between
processes, in the same way that it's possible to share them between threads.

> For example the design should treat also sharing/passing the CM 
> (RDMA-CM) ID among processes, and state the limitations, eg on the 
> private data etc.

This would have to be addressed in librdmacm. Let's finish libibverbs first.

> >> - Given that everything shared is in shared memory,
> 
> >I think we should try and keep shared memory usage to minimum.
> >For example, in mthca mr object just needs a key: we could
> >keep it in non-shared memory, just pass the key around
> >and save on shared memory usage.
> 
> what do you refer by "it" here?
> is it the lkey of the memory used for 
> the QP, or the lkey describing the rx/tx buffers?

Both, there's no real difference.

> On the latter case, looking on ib_umem_get, it uses current->mm etc,
> doesn't this mean that there should be some difference between shared to 
> non shared memory?

This is only used for registering the memory. Assuming it is registered
by some process, we can pass the key around between processes.

-- 
MST