[ofa-general] Re: Re: [PATCH RFC] sharing userspace IB objects

[Gleb Natapov]

On Tue, Jun 26, 2007 at 12:51:25PM +0300, Michael S. Tsirkin wrote:
> > Quoting Gleb Natapov <glebn at voltaire.com>:
> > Subject: Re: Re: [PATCH RFC] sharing userspace IB objects
> > 
> > On Tue, Jun 26, 2007 at 10:06:41AM +0300, Michael S. Tsirkin wrote:
> > > >    Is there really a strong use case for making every type of object
> > > >    shareable?  Can we handle the SRC stuff without going to this
> > > >    extreme of complexity?
> > > 
> > > This is not directly related to SRC: this is an effort
> > > to make it possible to share QPs, CQ etc across processes
> > > in the same way as they can be currently shared across threads.
> > > So assuming that we want multiple processes to post to
> > > the same QP, how can we support this?
> > 
> > Are you absolutely sure you even want to support this?
> 
> Take a look here :)
> http://www.quotedb.com/quotes/1007
So there is still a chance you'll reconsider :)

> 
> > What is the user case?
> 
> Use case? Scalability. Pls go over Dror's presentation given at Sonoma -
> he calls this SSQ.
As far as I can tell he is talking about HW supported solution and not
half baked SW one.

> 
> > If multiple processes what to post to the same QP how will you
> > ensure that right process will receive right completion event?
> 
> Same as with threads - memory for CQEs and locks will be allocated
> in shared memory to make it possible for multiple processes to poll
> CQ simultaneously, and they get completions in FCFS order.
> What to do with them is up to the user.
Are you going to use this API? How? There is no point in discussing user
API without specifying HOW user will be using it. You have to ask what
user want and design your API accordingly and not other way around. So
suppose I want to use proposed API to implement super scalable MPI. I
setup shared QP/CQ/... and each rank start to post into the QP and
receive completion from CQ and suppose rank A picked completion that
belongs to rank B so I will need to setup out of band channel to pass
this completion from A to B. This is not looks good at all to me.

--
			Gleb.