[openib-general] [PATCH] RDMA / IB CM: support immediately sending replies to received messages

[Sean Hefty]

Or Gerlitz wrote:
> 1) the CM moves the QP state to RTS before sending the REP and hence per 
> the QP its fine to do TX before getting the ESTABLISHED event.

Correct.

> 2) if a ULP calls rdma_establish (eg when polling an RX from CQ --> QP 
> --> un ESTABLISHED CMA ID or from the COMM_EST case of the qp async 
> event handler) it is ensured they will get ESTABLISHED event so its up 
> to them if to wait for the event before processing the RX or not.

Note that since the QP is in the RTS state, the IB COMM_EST event will _not_ be 
generated.  This is the key difference between these approaches.  The user would 
need to call rdma_establish when polling a RX from the CQ.  Failure to do so 
would result in a connection failure if the RTU were not received.  Calling 
rdma_establish would result in an RDMA CM ESTABLISH event.

> OK, fair enough. Personally i preferred the patch set that implemented 
> everything within the ib stack and just had this little requirement on 
> the ULP to hold on with doing TX before getting the ESTABLISHED event, 
> but this one makes sense as well.

My preference is to provide whichever solution makes it easier on the majority 
of the ULPs (ignoring spec changes at the moment).  I don't know which fix ULPs 
will find easier to work with.  Maybe we can come up with a compromise where we 
expose rdma_establish, but transitioning to RTS before the REP is sent requires 
that the ULP do the transition...?

> Also, i understand that for APM support a patch in the spirit of what 
> you were suggesting (ie track local QPNs and affiliated QP async events 
> in the CM) would be merged anyway, correct?

Correct.

- Sean