[ofa-general] Port space sharing in RDS

[Jon Mason]

During RDS init, rds_ib_init and rds_tcp_init will both individually bind to the
RDS port for all IP addresses.  Unfortunately, that will not work for iWARP for
2 major reasons.  

Firstly, the binding of a CM ID to IP address via rdma_bind_addr on INADDR_ANY
will cause the first caller to bind to all IP addresses/Devices (and the 
subsequent calls will fail).  So whichever module (IB or iWARP) that is called 
first will break the second (and cause the module loading to abort).  

Secondly, iWARP and the Linux stack must share the same port space.  If bound to
the same port, the RNIC will not be able to tell if the incoming RDS packet is 
for TCP or IB/iWARP RDS module.  It appears to be preferring the IB/iWARP RDS 
module and not passing the packet to the TCP RDS module.  Thus currently, iWARP 
adapters will not work if both TCP and IB are enabled in RDS.  

Regardless of whether iWARP support is separate or rolled into IB, these issues 
need to be resolved.  I am open to suggestions about how to go about correcting 
this.  One idea to correct the first issue, we can have the bind and all other 
device specific setup of both IB and iWARP handled by a single function which 
will then, based on node_type, handle the IB or iWARP case.  The second issue is
more complicated, as there is currently no way for the rdma_bind_addr to know if
the port is already in use and vice versa.  Obviously, we can make TCP/IWARP
inversely dependent on each other during compile time, but I'm not sure that is
a good long term strategy.

Thoughts?

Thanks,
Jon