[openib-general] Re: [PATCH 3/3] iWARP CM - iWARPConnectionManager

[Steve Wise]

On Wed, 2006-03-22 at 11:40 -0800, Sean Hefty wrote:
> >> For example, as soon as the user calls connect(), can they receive a CLOSE
> >> event, even before the connect() call returns?
> >
> >No.  connect results in a CONNECT_REPLY event always. Not a CLOSE
> >event.
> 
> What if the remote side sends the reply, then decides to abort or disconnect?

This would be seen by the local IWCM as CONNECT_REPLY(success) at which
point the QP is in RTS, and the cm_id is CONNECTED.  Then the reception
of an abort would generate a CLOSE event to the IWCM.  Or if the remote
side does an orderly shutdown, a DISCONNECT event is sent up when the
FIN arrives, and a CLOSE event is sent up when the connection shutdown
finishes.

I'm speaking here from the provider's perspective...


> I'm considering a case where multiple events may be queued.  I.e. after calling
> connect(), what events can a user see without making any other calls?
> 

The user, being the IWCM, will always see a CONNECT_REPLY to a connect
downcall.  

> >> If so, are there any issues
> >> here?  Is it possible for the user to call down to the provider, after the
> >> provider has generated a CLOSE event, resulting in accessing the wrong
> >> connection, or crashing in the provider?
> >
> >The IWCM prevents this, methinks, by failing any downcalls once the
> >cm_id is no longer in a CONNECTED state.
> 
> The issue is that a state check could succeed, with the downcall pending when an
> event is received that changes the state.  The downcall cannot be stopped at
> this point, and will hit the provider's code.  Maybe the call/event model
> prevents this from ever being an issue?
> 

I _think_ the IWCM prevents this.  Tom?  

> >For orderly close, the provider needs to wait until the close completes
> >or is aborted due to protocol errors.  This can be implemented in many
> >ways, but blocking the caller is perhaps the simplest.  Also, note, that
> >the QP needs to stay around until it transitions out of CLOSING by the
> >provider.  This further complicates things and is different from IB.
> 
> As long as the blocking is reasonably short, this sounds fine.  I'm thinking
> more of the case where a module is trying to release its resources before being
> unloaded.

For a module unload, the connections should be aborted, not
disconnected.  Aborts are quick.  Disconnects can take a long time
because its end-to-end, and if connectivity is lost during this
end-to-end operation, TCP timeout mechanisms will be used to decide when
to "give up".

> 
> >connect(), listen(), and when incoming iwarp connections get
> >established.
> 
> I didn't consider that last case.  The provider creates the context at this
> point, but when is it associated with the cm_id?  On accept()?  

Yes.

> How are
> additional events that are generated in this case handled, or is that even
> possible?  I.e. you've notified the user of a new connection request, when the
> connection is aborted.  The user hasn't called accept() or reject() yet.
> 

The provider must handle this and _not_ pass any events up until the
IWCM has accepted or rejected the connection request.  In the case of an
connect request upcall, followed by a connection abortion, followed by a
accept_cr downcall, the accept_cr downcall would return an error like
-ECONNRESET.  OR the provider could allow the accept_cr to succeed,
which moves everything to RTS/CONNECTED, then generate a CLOSE upcall.
Either way is correct methinks.  I think providers would do the former.


Steve.