[openib-general] design for communication established affiliated asynchronous event handling

[James Lentini]

On Thu, 15 Jun 2006, Sean Hefty wrote:

> >The cma/verbs consumer can't just ignore the event since its qp state is
> >still RTR which means an attempt to tx replying the rx would fail.
> 
> In most cases, I would expect that the IB CM will eventually receive the RTU,
> which will generate an event to the RDMA CM to transition the QP into RTS.  This
> is why I think that the event can safely be ignored.  It does however mean that
> a user cannot send on the QP until the user sees RDMA_CM_EVENT_ESTABLISHED.
> 
> >I suggest the following design: the CMA would replace the event handler
> >provided with the qp_init_attr struct with a callback of its own and
> >keep the original handler/context on a private structure.
> 
> This sounds like it would work.  I don't think that there are any events where
> the additional delay would matter.
> 
> As an alternative, I don't think that there's any reason why the QP 
> can't be transition to RTS when the CM REP is sent.  

I like this idea. It simplifies how ULPs handle this issue. Are there 
any spec. compliance issues with this?

> A user just can't post to the send queue until either an 
> RDMA_CM_EVENT_ESTABLISHED, IB_EVENT_COMM_EST, or a completion occurs 
> on the QP.  (This doesn't change the fact that the IB CM still needs 
> to know that the connection has been established, or it risks 
> putting the connection into an error state if an RTU is never 
> received.)

If the passive side CM doesn't receive an RTU, the passive side CM 
should retransmit the REP. At least that is how I read 12.9.8.6 
"Timeouts and Retries" in the IBTA spec. I can't find where this 
happens in the code. Did I miss it?