[ofa-general] Re: [PATCH RFC] sharing userspace IB objects

[Caitlin Bestler]

general-bounces at lists.openfabrics.org wrote:
>> Quoting Roland Dreier <rdreier at cisco.com>:
>> Subject: Re: [PATCH RFC] sharing userspace IB objects
>> 
>> Some initial reaction, in no particular order:
>> 
>>  - Having to allocate everything in memory that the library mmap()s
>>    adds a lot of yucky stuff -- basically we need to implement our
>>    own allocator for the shared memory offets.
> 
> Right.
> 
>>    I guess we could wrap this
>>    in libibverbs and only implement it once but still we're basically
>>    reimplementing malloc().
> 
> Right.
> 
>>    Is there really a strong use case for making every type of object
>>    shareable?  Can we handle the SRC stuff without going to this
>>    extreme of complexity?
> 
> This is not directly related to SRC: this is an effort to
> make it possible to share QPs, CQ etc across processes in the
> same way as they can be currently shared across threads.
> So assuming that we want multiple processes to post to the
> same QP, how can we support this?
> 

Sharing QPs and CQs ultimately means sharing Protection Domains
and Memory Regions across processes. So obviously this would never
be a default. Basically you would be enabling a group of processes
to share Memory Regions, QPs, etc all created with a single PD.
The easiest way to support this in the hardware is to simply not
be aware that it is happening, that is, to treat all the processes
as though they were just threads.

I suspect that makes the prospective pool of users quite small. But
there are lesser sharings that could be of value:

	Passing Connection Requests to other processes, and allowing
	them to accept the connection.

	Passing an empty QP to another process, which could then
re-attach
	it to its Protection Domain and supply new memory for the SQ and
RQ.