[openib-general] CMA issue: bind selects the same port after close

[Michael S. Tsirkin]

Quoting r. Sean Hefty <mshefty at ichips.intel.com>:
> Subject: Re: [openib-general] CMA issue: bind selects the same port after close
> 
> Michael S. Tsirkin wrote:
> > We have encountered an issue in CMA: if
> > I bind to port 0, destroy the id, then bind to port 0 again
> > I often get back the same port from both binds.
> > 
> > TCP behaves differently - it seems to assign  new port numbers
> > each time.
> > This is an issue for some socket programs that assume that
> > the same port number won't be reused to a remote side that
> > connects to the same port after I have  closed by socket will get
> > connection refused message.
> > I also see applications looking for a port number that matches
> > some rule by repeating the create/bind/close cycle.
> > With CMA they always get back the same port number it seems.
> > 
> > Is this something that can be fixed in CMA?
> 
> I think we can fix this without a huge impact.  Is there anything that states 
> the way bind is supposed to behave wrt this?

I don't think so. But since that's how it works on linux and other systems,
apps assume this.

> Is there some delay between 
> releasing a port and it being re-used that needs to be taken into account?

TCP keeps port busy while in timewait state, unless REUSEADDR is given.
I have not yet seen any app rely on this, so it might not be important
to emulate this.

> The basic problem in the CMA is in cma_alloc_port().  If the port number (passed 
> in as snum) is 0, the first available port starting at 
> sysctl_local_port_range[0] is used.  We could instead start our search by 
> adding an increasing counter or a random value to the lower-end of the port 
> range.  Then expand the code to handle searching below our starting value if we 
> failed to find one above it.

Sounds good.

> Are the port numbers assigned by TCP sequential or more random?

TCP ports seem to be sequential.

-- 
MST