[openib-general] Re: Re: SDP_CONN_LOCK

[Libor Michalek]

On Sun, Feb 20, 2005 at 03:16:40PM +0200, Michael S. Tsirkin wrote:
> Quoting r. Libor Michalek <libor at topspin.com>:
> >   And here I would have thought SDP_CONN_HOLD as a macro would have
> > bugged people more. :)
> > 
> > -Libor
> > 
> 
> Regarding conn_hold/conn_put (macro or not):
> 1. conn_put is sometimes called while socket is locked (sdp_cm_connect),
>    sometimes not (sdp_ip2pr_path_complete). What gives?
>    Is it OK to destroy the socket while its locked? Seems strange ...

  conn_put() should not be called from inside the lock as you note. The
only reason that this has not caused a problem in this instance is that
the conn_hold() in that function, and conn_put() on an error, are always
the second reference on the connection.

  There are essentially four references which are being tracked by the
connection:

  - Socket interface. 
      Increment on socket creation. Either socket() system call or when
      a passive connect request is added to a listen sockets accept queue.
      Decrement on socket release. When close() is called on either the
      socket or a listen socket, in which case the accept queue is flushed.
  - CM interface.
      Increment on REQ send or receive.
      Decrement on CM IDLE event or right before an error is returned
      from the CM event handler
  - Address resolution.
      Increment on request.
      Decrement on completion.
  - Event execution. (CQ or CM)
      Increment using sdp_conn_table_lookup()
      Decrement before leaving event handler.

> 2. When conn_put is called *outside* the lock, what prevents conn_hold
>    from being called while conn_put is in progress, thus hold will be
>    acting in parallel with destroy?

  There should only ever be contention in the case where the request
for the reference count is comming from sdp_conn_table_lookup(), but
you are right there is a small race during conn_put and before the
connection is removed from the table. 

-Libor