[openib-general] Re: Re: port_num

[Steve Wise]

> I don't think duplicating the policy works does it? Won't the CMA _and_
> the host stack will be handing out ports from the same port range, and
> inevitably, they will collide?
> 
> - Kernel mode sockets app with CAP_NET_BIND_SERVICE calls connect() to
> 10.10.105.10:2049, 
> - Stack hands out port from reserved range (1023)
> - CMA app with CAP_NET_BIND_SERVICE calls rdma_connect to
> 10.10.105.10:2049
> - CMA hands out port from reserved range (1023)
> - Now you have two different apps with the same 4-tuple. 
> 
> Somehow the host stack and the CMA need to be reserving ports from the
> same pool or there will be collisions.
> 
> UNLESS you keep the force the local IP addresses to be different, but
> that's another story...and has it's own issues ...
> > 
> > 

Just to clarify:  Tom is talking about iWARP devices here that support
the native stack -and- the rdma stack with the same offload device.  For
IB devices, the above "collision" doesn't matter because IB really
doesn't use the ip addresses at all after the RC is setup.  AND, the
native stack TCP support in IB (IPoIB) uses a different QP and service
type (UD). So there's no problem with said collision.  

For an iwarp device, however, this issue exists.  In case its not
obvious, the problem is that the RNIC adapter needs to know based on
part or all of the 4-tuple in an incoming packet, whether this is an
RDMA or native stack TCP packet.  So the ports/ipaddrs are used as the
demux.  An application listening on a port/ipaddr pair is either a
sockets app using the native stack -or- an RDMA app.  It can't support
both. 

The chelsio driver suffers from the problem.  The current workaround is
that you use a different ip address for the rdma connections (or a
different entire subnet).  

Also, the chelsio provider CM picks local ephemeral ports if the local
port passed down in a listen_create or connect is zero.  It does _not_
integrate with the native tcp port space.  

BTW:  IMO, This part of the chelsio provider CM needs to move up into
the iwarp CM eventually (after we get the initial "phase 1 iwarp
support" in the main trunk so we don't spend our time continually
merging with the main trunk).  The IWCM needs to manage ephemeral
ports...either sharing the native stack tcp port space or not.  But it
needs to manage ports for all iwarp devices to keep the ephemeral port
numbers consistent across devices.

Steve.