[ofa-general] Re: Re: [PATCH RFC] sharing userspace IB objects

[Michael S. Tsirkin]

> Quoting Hal Rosenstock <halr at voltaire.com>:
> Subject: Re: [ofa-general] Re: Re: [PATCH RFC] sharing userspace IB objects
> 
> On Mon, 2007-07-02 at 08:58, Dror Goldenberg wrote:
> > Hal Rosenstock wrote:
> > > On Sun, 2007-07-01 at 15:05, Gleb Natapov wrote:
> > >   
> > >> On Sun, Jul 01, 2007 at 07:27:24PM +0300, Dror Goldenberg wrote:
> > >>     
> > >>>> SSQ is needed for scalability, no need to explain this (by 
> > >>>> the way RD is needed for the same reason too. What's Mellanox 
> > >>>> plan to support it?
> > >>>>         
> > >>> RD is not supported in hardware today. Implementing RD is extremely 
> > >>> complicated. To solve the scalability issues on MPI like applications
> > >>> we believe that SRC and SSQ are the right solutions. It is much simpler
> > >>> for implementation by both software and hardware. By MPI-like I refer
> > >>> to applications that have some level of trust between two processes of
> > >>> the
> > >>> same application. RD also has some performance issues as it only 
> > >>> supports one message in the air. Those performance issues are solved
> > >>> by design in SRC/SSQ.
> > >>>
> > >>>       
> > >> Didn't know about RD limitation. Is this shortcomings of IB spec or
> > >> general limitation of reliable datagram? RD looks much nice to me then SRC/SSQ.
> > >>     
> > >
> > > I think Dror is referring to number of messages in flight per EEC and
> > > number of messages in flight per QP being limited to 1 per IBA spec.
> > > Number of messages enqueued per EEC/QP is implementation dependent.
> > >
> > > -- Hal
> > >   
> > Correct. The number of messages in flight per EEC is 1 per IB spec.
> > The fact that IB requires SQ WQEs to complete in order, even if their 
> > destination is different EECs,
> 
> Where's this requirement in the spec (and could this be relaxed as it
> seems like it is overly "specified") ? Just wondering...

For example:
	10.8.5 RETURNING COMPLETED WORK REQUESTS

	...

	Except for RD Receive Work Queues and Receive Work Queues associ-
	ated with an SRQ, Work Completions are always returned in the order
	submitted to a given Work Queue with respect to other Work Requests on 
	that Work Queue.

-- 
MST