[openib-general] Re: [PATCH] CMA: allow/require bind beforeconnect

[Tom Tucker]

On Tue, 2006-03-28 at 09:33 -0600, Steve Wise wrote:
> On Mon, 2006-03-27 at 14:40 -0800, Sean Hefty wrote:
> > >On IB, it also maps ports to service IDs.
> > >
> > >This argues that the port values are within the scope of the CMA
> > >and that the active side behavior should be changed.
> > 
> > The passive side maps the source port, and the active side maps the destination
> > port.  So, a port assignment is only needed on the passive side.  Although, if
> > code needs to be there to support listens that don't care which port they listen
> > on, then using it on the active side isn't any more difficult.
> > 
> 
> As Tom and James pointed out, NFS over RDMA needs the port number of
> each client to do security policy.  And it is expected that the client
> will be assigned a privileged port only if the client process is
> privileged.  And privileged ports are < 1024.  So the active side also
> maps its local port in some instances.

Specifically, if an app wants to specify the local port, it calls bind()
prior to connect() and only privileged apps can allocate ports below
1024. This equates roughly to a CMA app specifying a non-zero sin_port
value in the route.addr.src_addr.sin_port when calling
rdma_resolve_addr.

> To support NFS, the CMA needs to manage local port assignment.  And to
> support applications that are RDMA transport independent, I assert that
> the CMA should allocate port numbers using a consistent model across all
> rdma transport types.   Otherwise, you break NFSoRDMA methinks. And that
> model should be akin to the native TCP stack's model for port
> allocation. 

Specifically, if route.addr.src_addr.sin_port == 0 in rdma_resolve_addr,
the CMA should reserve a port from the port space starting at 
sysctl_local_port_range[0] and ending at sysctl_local_port_range[1]. 
If the source port number is NOT zero, it should reserve the specified port.
If the requested port is below 1024 and the consumer is NOT, then 
rdma_resovle_addr should fail with -EPERM. If the requested port is already 
in use rdma_resolve_addr should return -EADDRINUSE. Otherwise, we're good.

The bulk of this logic is in inet_csk_get_port. The security policy is 
in inet_bind.

> 
> 
> > An application wants to connect to some server, and doesn't care what port
> > number is used.  Why does it care that the source port assignment is unique?
> 
> The app _usually_ doesn't care.  See NFS discussion for a client app
> that does care.  Also, providers DO care.  Because of this issue, the
> chelsio iwarp provider right now has to allocate its own ephemeral ports
> at connect time.  This logic should be moved into the IWCM or maybe the
> CMA and an explicit bind() operation be exported by the iwarp providers
> to allow the IWCM or CMA to track all port allocations.
> 
> > And how unique is unique?  System wide?  Within the RDMA CM?  Within all of TCP?
> > Within all TCP ports over a specific IP address?  Within the RDMA CM within its
> > TCP space over a specific IP address?  Etc.
> > 
> > What scope is needed, and why?  SDP has its own address family.  Does this give
> > it its own port space, or does it need to share it?
> > 
> 
> This a harder question.  I think it should be at least global across the
> RDMA CM.  Why?  Because that's the model that is expected by IP
> applications that are being moved over to RDMA/CMA.
> 

In Linux, this is controlled by the hashinfo structure passed to the
inet_csk_get_port function. A similar strategy should be employed in the
CMA. This will allow us to tweak policy (new port domains) by trivially
creating or combining port spaces in hashinfo structures.

> My 2 cents.
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general