[openib-general] Re: port_num

[Caitlin Bestler]

openib-general-bounces at openib.org wrote:
>> I'd like to suggest that CMA implement its own rule along the lines
>> of: if port is 0, bind to a port number unused by CMA in this port
>> space and on this device.
> 
> To clarify: a user is permitted to bind to a specific port,
> but use a wildcard IP address.  Just because a user binds to
> a port doesn't mean that they also bind to an RDMA device.
> This permits a single listen across all RDMA devices.
> If one of those devices is an iWarp device, then the port
> number selected by the CMA ends up needing to be a true TCP
> port number.
>

Listening on a port across a mixture of RDMA and non-RDMA
devices (or even dissimiliar RDMA devices) is quite problematic.
The CONNECT_REQUEST events returned ARE bound to a specific
device, and must be accepted/rejected to that device.

Mixing a host stack listen with a CMA listen is particularly
complex because the connection request will be delivered to
the consumer by a different method.

These are the reasons why the RDMAC originally specified that
the host stack would create *all* connections, and then some
would be migrated to RDMA mode. Unfortunately the netdev
team does not share that vision. Respecting that, we basically
have to decide in an inbound connection request for a given
IP Address / Port number received on an Ethernet port of an
iWARP RNIC will be handled by the iWARP stack or forwarded
to the host stack. You really have to do one or the other,
or convince netdev to accept the "modify qp to rts with 
llp socket handle" strategy.

Given the need to post receive buffers before accepting
a connection, it is actually not all that feasible to have
an RDMA application that is ready to accept connections
from *any* RDMA device. The application has to know what
the RDMA device are in advance, create protection domains
on each, and probably create memory regions on each. It may
even want to pre-allocate QPs. It is certainly not going to
respond to a connection request from "eth7" by finding out
what the RDMA device for eth7 is, opening it, creating the
PD, MR and QP and *then* accepting the connection request.