[openib-general] MAD queuing model

[Roland Dreier]

    Sean> Does anyone have any thoughts about where MAD queuing should
    Sean> occur?  If a work request for a MAD cannot be immediately
    Sean> posted to a QP, should a call to ib_mad_post_send fail, or
    Sean> should the work request be queued for later?  Along this
    Sean> same line, should a MAD requiring RMPP post multiple work
    Sean> requests or post a single request at a time, until it
    Sean> completes?

I definitely think the queueing should be done in the access layer so
that consumers don't have to deal with the send queue full condition.
(Otherwise we would have to call consumers back when space was
available, and there still wouldn't be a way to ensure fair queueing
and avoid the possibility of starving one consumer indefinitely).

For RMPP, we could have the RMPP layer wait for each send to complete
before posting another send.  However I'm not sure this gains much and
it seems simpler just to let the RMPP queue all its sends as they are
ready and let the MAD layer handle posting as many sends as will fit
in the work queue.

 - R.