[ofiwg] send/recv "credits"

[Jason Gunthorpe]

On Wed, Sep 24, 2014 at 09:16:25PM +0000, Hefty, Sean wrote:
> > Well, I don't see how you can expect an application to efficiently
> > manage message flow to hidden queues. At a minimum an app needs to
> > know messages can be submitted, know when the hidden queue is full and
> > be able to sleep until it becomes !full.
> 
> A viable solution is to require providers to support an infinite
> queue, subject to memory constraints.  I don't think MPI limits what
> an application can queue.  I doubt everyone will be happy with the
> potential performance impact of this, but then again, the request is
> going to be queued somewhere...

I've never heard of a networking layer with 'infinite' buffering. Is
that even really workable? That sounds like it leads to OOM and/or
deadlock.

How does an app know to switch from a sending mode to a listening mode
without a backpressure signal on the sendq?

> I'm actually more concerned about credits at the receive side.  :)

How so? Do you want to abstract multiple HW recieve queues into one SW
queue? That sounds incompatible with app mediated credit schemes?

Jason