[openib-general] MAD queuing model

[Hal Rosenstock]

On Wed, 2004-09-01 at 12:28, Sean Hefty wrote:
> Does anyone have any thoughts about where MAD queuing should occur?  

I am just thinking "out loud" in my answers below.

> If a work request for a MAD cannot be immediately posted to a QP, 
> should a call to ib_mad_post_send fail, or should the work request 
> be queued for later?  

It could be put on a deferred send queue that would be processed when
the next send completion occurs. This would make the access layer
implementation a little more complicated but save the client from
rerequesting.

> Along this same line, should a MAD requiring 
> RMPP post multiple work requests or post a single request at a time,
> until it completes?  (By completion, I mean the work request only, 
> and not an RMPP response.)

Either (up to client) but there may be timing considerations if the 
WRs associated with an RMPP MAD are posted separately. Meaning that
in the case where some WRs were posted and others deferred, some RMPP 
timeout could occur. It could cause a transaction too long and an 
ABORT would be received in response. Are there others ? It seems safer
to post all WRs associated with an RMPP MAD as close together as
possible (ideally it would be "atomic").

> My initial thoughts are to queue the MADs in the access layer.  
> But timers would not start until the work request had actually 
> been posted.

I agree that if deferred sending were to be done, request/response
timeouts (ib_mad_send_wr.timeout_ms) should not start until the MAD is
actually posted. 

This brings another question to mind. Should timeout_ms be ignored for
send methods if supplied in the send WR ?

-- Hal