[openib-general] Re: [PATCH 3/3] iWARP CM - iWARPConnectionManager

[Steve Wise]

On Wed, 2006-03-22 at 13:09 -0800, Caitlin Bestler wrote:
> openib-general-bounces at openib.org wrote:
> >>> For example, as soon as the user calls connect(), can they receive a
> >>> CLOSE event, even before the connect() call returns?
> >> 
> >> No.  connect results in a CONNECT_REPLY event always. Not a CLOSE
> >> event.
> > 
> > What if the remote side sends the reply, then decides to abort or
> > disconnect? I'm considering a case where multiple events may be
> > queued. I.e. after calling connect(), what events can a user see
> > without making any other calls?
> > 
> 
> One of the differences here is that for iWARP there is a natural
> ordering of events that is unambiguous. TCP connection follows
> a certain sequence, and after that there is the TCP sequence
> itself (or the equivalent SCTP features).
> 
> So if the remote side sends the reply and *then* decides to
> abort or disconnect that step will unambiguously be an abort
> or disconnection of an *established* connection (because the
> sequence numbers will be greater). There isn't the problem
> that IB can face where a disconnect might be received before
> the connection acceptance.


Another way of stating this is: connection establishment/teardown is
done in-band in the TCP connection vs for IB, its done out-of-band with
respect to the RC connection being established.  In fact, the IB uses a
_different_ QP for the connection setup/teardown.  iWARP uses the SAME
QP (at least for connection teardown).

> 
> So the only possible race conditions are between locally
> initiated events and remote ones. These have been worked
> through in forums such as RDMAC, DAPL and RNIC-PI. It relies
> on atomic state transitioning, but it otherwise simple.
> I'm not claiming to have done a by-hand software proof,
> but I haven't spotted anything in IW-CM that contradicts
> my understanding of the relevant state machines.
> 
> A formal state model is a bit tricky to present because
> there are multiple objects involved on the passive side
> in each connection establishment. By the time you are
> accurate the state model is no longer informative to
> a first time reader.