[ofa-general] Re: [PATCH RFC] sharing userspace IB objects

[Jeff Squyres]

On Jul 2, 2007, at 5:15 PM, Galen Shipman wrote:

> While I think the SRC design makes sense I also have concerns  
> regarding SSQ.
> As Gleb has pointed out, if the hardware doesn't do the demux then  
> the application has to.  It sounds like there are two proposals to  
> deal with this hardware limitation in software (sigh).
>
> 1) Process A polls CQ, if WQE belongs to Process B, Process A will  
> drop the WQE in a shared memory region that Process B  will poll.  
> [snip]
> 2) Process A peeks CQ, if WQE belongs to Process B, it doesn't  
> process it [snip]
>
> In my opinion the demux belongs in the hardware, otherwise we end  
> up complicating an already complicated code base to support a  
> feature which unless I am missing something will have no benefit to  
> real applications.

I agree.  I cannot see how SSQ will be useful in Open MPI -- it makes  
the code *much* more complicated and effectively guarantees to add  
latency for the common case.  I don't see how to explain it better  
than Gleb/Galen already did.

If Mellanox wants to implement SSQ for other reasons, fine.  But  
based on the explanations so far, I don't see us using it in [Open] MPI.

-- 
Jeff Squyres
Cisco Systems