[openib-general] MAD queuing model

[Sean Hefty]

On Wed, 01 Sep 2004 15:56:24 -0700
Roland Dreier <roland at topspin.com> wrote:

>     Michael> I'd like to propose a simpler approach: if the Q is full,
>     Michael> drop the MAD without any indication to the user (pretend
>     Michael> the MAD was sent), and let RMPP retry.
> 
> It's an interesting idea (sort of the most trivial possible queue
> scheduler).  However I don't see how you handle the fact that the
> consumer will be waiting for a send completion to free the MAD, and I
> think the consumer will rely on completions occurring in order.

I thought about this approach as well, and I think it could work (and probably fairly well for small to medium configurations).  I'm not sure that I would consider it a "best effort" approach, but avoids a lot of complexities trying to queue the MADs.  It also puts less stress on the fabric and remote receive queues, avoids excessively long queue times, and consumes fewer system resources.  (Maybe I undervalued this approach...)

We could define the MAD interface such that the completions could come in any order.  That would give us the most flexibility regarding queuing.  I think out of order completions is necessary for MADs that require responses anyway.