[openib-general] Re: [PATCH] CMA: allow/require bind before connect

[Tom Tucker]

On Sun, 2006-03-26 at 18:46 +0200, Michael S. Tsirkin wrote:
> Quoting r. Tom Tucker <tom at opengridcomputing.com>:
> > Subject: Re: [PATCH] CMA: allow/require bind before connect
> > 
> > On Sun, 2006-03-26 at 17:34 +0200, Michael S. Tsirkin wrote:
> > > Quoting r. Tom Tucker <tom at opengridcomputing.com>:
> > > > Subject: Re: [PATCH] CMA: allow/require bind before connect
> > > > 
> > > > On Sun, 2006-03-26 at 16:59 +0200, Michael S. Tsirkin wrote:
> > > > > Sean, CMA currently bails out if I bind to ANY address before connect.
> > > > > I think in TCP you bind before connect 
> > > > 
> > > > It is not typical to use bind on the active side before connect. When it
> > > > is done, is when the application wants to control which local port
> > > > and/or interface to use when connecting to a remove peer. bind(2) is
> > > > typically used on the passive side before listen.
> > > 
> > > Tom, I have a connection bind on a local port to control which local port
> > > to use when connecting to a remove peer.
> > > 
> > > Further, I think TCP connections autobind to a local port on connect,
> > > even if you don't do it explicitly.
> > > I think a local port is required for the communication to work in TCP.
> > > 
> > > At the level I work in SDP, Linux net core calls bind before connect if the
> > > socket source port is 0. I could override this but I don't think its a good
> > > idea since the point is to emulate TCP: TCP connections always have a source
> > > port and so should CMA connections.
> > 
> > Wow. Now I'm really lost. I thought you were trying to specify the local
> > port. You're not, you're asking the stack to select one for you -- which
> > is the common case. 
> 
> Yes, I am.
> 
> > I'm still searching for the actual problem you're having. Are you saying
> > that the local port number after connect on SDP over IB is 0?  If so,
> > then something is fundamentally busted. If sin_port is 0, the stack is
> > supposed to allocate an ephemeral port for you. That's what 0 means. 
> 
> CMA does not allocate ephemeral ports currently.
> This actually makes some sense since address families/port spaces expect
> different semantics wrt which port numbers are legal.

I'm not getting the families/port spaces argument. We're supposed to be
emulating TCP/IP here. I propose that, to the extent possible, we behave
like the Linux stack. 

Some applications (NFS) qualify the remote peer based on port (in this
case, the port number the CMA should be allocating).  I just confirmed
that the remote port validated by NFS is always 0 for IB clients. 

> 
> > The bind in the net/core case is specifically for this purpose, to cause
> > the stack to allocate a port number. In the CMA, the rdma_resolve_addr
> > call has everything you need. It is sort of an uber-bind.
> 
> In TCP, if you don't bind a local port you'll be autobound on connection.
> I don't know whether its necessary to have this in CMA: kernel
> applications can easily bind before connect.
> But socket applications also expect the ability to bind and be able to
> get port number and verify at that time that the port is available.
> 
> 
> > The port
> > number in the src_addr should be being used to either confirm that that
> > the specified local port is available (and you have rights to the
> > specified port if it is below 1024) and if it is zero, allocating one
> > for you. I think the only question is 'when' this allocation/checking is
> > done. It could be done early in the rdma_resolve_addr, or delayed until
> > rdma_connect time. I think the former is preferred.
> 
> If CMA does autobind, we need it at rdma_resolve_addr time.

Agreed. Also, when the client specifically requests a local port number,
the allocation needs to take into account the security policy. 

> 
> > > Since bind to port 0 isn't supported in CMA yet, I proposed making an
> > > explicit bind a requirement before resolve_addr - not a serious
> > > overhead and it keeps the code simple.
> > > 
> > 
>