[ofa-general] Re: Re: [PATCH RFC] sharing userspace IB objects

[Michael S. Tsirkin]

> Quoting Dror Goldenberg <gdror at dev.mellanox.co.il>:
> Subject: Re: [ofa-general] Re: Re: [PATCH RFC] sharing userspace IB objects
> 
> Hal Rosenstock wrote:
> >On Sun, 2007-07-01 at 15:05, Gleb Natapov wrote:
> >  
> >>On Sun, Jul 01, 2007 at 07:27:24PM +0300, Dror Goldenberg wrote:
> >>    
> >>>>SSQ is needed for scalability, no need to explain this (by 
> >>>>the way RD is needed for the same reason too. What's Mellanox 
> >>>>plan to support it?
> >>>>        
> >>>RD is not supported in hardware today. Implementing RD is extremely 
> >>>complicated. To solve the scalability issues on MPI like applications
> >>>we believe that SRC and SSQ are the right solutions. It is much simpler
> >>>for implementation by both software and hardware. By MPI-like I refer
> >>>to applications that have some level of trust between two processes of
> >>>the
> >>>same application. RD also has some performance issues as it only 
> >>>supports one message in the air. Those performance issues are solved
> >>>by design in SRC/SSQ.
> >>>
> >>>      
> >>Didn't know about RD limitation. Is this shortcomings of IB spec or
> >>general limitation of reliable datagram? RD looks much nice to me then 
> >>SRC/SSQ.
> >>    
> >
> >I think Dror is referring to number of messages in flight per EEC and
> >number of messages in flight per QP being limited to 1 per IBA spec.
> >Number of messages enqueued per EEC/QP is implementation dependent.
> >
> >-- Hal
> >  
> Correct. The number of messages in flight per EEC is 1 per IB spec.
> The fact that IB requires SQ WQEs to complete in order, even if their 
> destination is different EECs, makes it pretty challenging to have an 
> implementation that can really process more than one message 
> simultaneously per QP.

Hmm, I guess this requirement could easily be relaxed - in a way
similiar to what was done for SRQ - without breaking applications.
WRID is sufficient to identify the WR even without ordering guarantees.

-- 
MST