[openib-general] Re: RFC: cma: need rdma_unbind

[Tom Tucker]

On Tue, 2006-04-25 at 00:19 +0300, Michael S. Tsirkin wrote:
> Quoting r. Tom Tucker <tom at opengridcomputing.com>:
> > Subject: Re: RFC: cma: need rdma_unbind
> > 
> > On Mon, 2006-04-24 at 11:15 -0700, Sean Hefty wrote:
> > > Michael S. Tsirkin wrote:
> > > > Sean, it seems that a rdma_unbind API is necessary for SDP SO_REUSEADDR support.
> > > > This should only remove association between the port and the id, without
> > > > affecting the CM state.
> > > > 
> > > > Reason: when SO_REUSEADDR is set, the port becomes usable immediately
> > > > after close call, but the QP must not be closed immediately -
> > > > rather its closed after graceful close or after timeout.
> > > > 
> > > > Thus I can not destroy the id, but the association with
> > > > port needs to be cancelled.
> > > 
> > > I'm in the process of adding rdma_set_option and rdma_get_option calls.  If 
> > > SO_REUSEADDR support were added through these calls, can we come up with a 
> > > solution that doesn't require adding an rdma_unbind call?
> > 
> > There is no need for an unbind call -- the sockets code, for example,
> > doesn't work this way. Instead, it looks at a 'reuse' flag and decides
> > whether to allow a subsequent bind or use-port request based on the
> > value.
> 
> I just tested this and this does not seem to be how it works:
> if the socket is in TIME_WAIT state, SO_REUSEADDR let's you bind another
> one to this port.
> If the socket is not in TIME_WAIT state, you still get EADDRINUSE.

The code snippet is taken out of a stack that makes all the checks that
you refer to. My point is that the CMA doesn't need another API
applications call to explicitly release ports because they are
implicitly released when the app calls rdma_disconnect and/or
rdma_destroy_id. What I thought we were discussing was controlling how
quickly the port can be reused through configuring the REUSEADDR
option. 

For IB, I would think the option is effectively a no-op since the
TIMEWAIT state is a very short lived affair, not the 2MSL lifetime of a
TCP/IP connection.

In any event, it is my opinion that the policy for how ports are
released belongs in the stack (the CMA in this case), not in the
application where it is controlled with an API like rdma_unbind.

Are you suggesting something else and I'm just missing it?

> 
> socketfaq seems to agree
> http://www.unixguide.net/network/socketfaq/4.5.shtml
> 
> And CMA does not know about socket state - that's managed by ULP.
> 
> 
>