[openib-general] design for communication established affiliated asynchronous event handling

[James Lentini]

On Fri, 16 Jun 2006, Or Gerlitz wrote:

> On 6/15/06, James Lentini <jlentini at netapp.com> wrote:
> > ib_cm_establish() doesn't emulate an RTU reception. It generates an
> > IB_CM_USER_ESTABLISHED event (not an IB_CM_RTU_RECEIVED event). The
> > CMA's cma_ib_handler() doesn't recognize a IB_CM_USER_ESTABLISHED
> > event. The QP's state will not be moved to RTS.
> 
> This is what i was suspecting, Sean can you confirm that? if it does
> not emulate RTU
> reception, than what it does do?
> 
> > Consumers don't actually have to queue the completions, they have to
> > defer posting sends (either in response to the recvs or otherwise)
> > until the QP moves to RTS. Could the implementations queue up the
> > requests for the consumers?
> 
> nope the CM/CMA are not in charge of the consumer CQ, so there is no 
> way for them to queue those completions and anyway, i think its 

I was refering to requests, not completions. In any event, I like 
Sean's idea of moving the QP to RTS when a REP is sent better.

> wrong for lower layer to queue completions, this "race" exists by 
> IB's nature (since the RTU goes to QP1 and the data to the user's QP 
> and the two QPs are totally unrelated) so if you want to have 
> production with IB you need to handle this case in your code, as 
> others do.

Agreed.

> > Strictly speaking, IB requires an error to be generated (C10-29 in 
> > the IBTA spec. vol 1, page 456). Still, it would be nice if 
> > consumers didn't have to be worry about this issue.
> 
> What do you mean by error, this async event happens all the time, 
> you can't error the establishment just b/c it happend. I don't have 
> access now to the spec, so i can't say what i understand from the 
> section you have pointed to.

Again, I was refering to requests, not completions.