[openib-general] RDMA connection and address translation API

[Roland Dreier]

    Tom> The issue is that this connection will be established when
    Tom> the server may only want to accept requests that are
    Tom> targetted to the 10.10.1.1 address.  I don't get why this is
    Tom> such a big deal. You can preclude this behavior by simply
    Tom> keeping a one to one mapping between the IPv4 addresses and
    Tom> the GIDs using the existing protocols and without mandating a
    Tom> private data format across *all* ulps and transports.

Well, a few problems with what you say:

 - ATS does not help at all with the case of a multi-homed interface.
   Unless the remote system puts the IP it's trying to connect to
   somewhere in the connection request, there is no way to be psychic
   and recover this information.

 - Mandating ATS use is dictating protocol design just as much as
   requiring the CM private data to carry source and destination IP
   addresses.

 - It's not just preventing connections to the wrong local address.
   NFS-RDMA wants the remote source address (ie getpeername()) so that
   it can look it up in the exports list.

 - Saying that a given GID may only have a single IP address is
   definitely a case of the cure being worse than the disease.  I
   don't think we can forbid perfectly valid multi-homed
   configurations just because it's inconvenient for us to support them.

By the way, as far as I can tell, there is NO formal documentation of
the NFS-RDMA wire protocol.  The current draft (draft-ietf-nfsv4-rpcrdma-01.txt)
simply says:

     This protocol is designed to function with equivalent semantics
     over all appropriate RDMA transports.  In its abstract form, this
     protocol does not implement RDMA directly. [...]  It therefore
     becomes a useful, implementable standard when mapped onto a
     specific RDMA transport, such as iWARP [RDDP] or Infiniband [IB].

     [...]

     In setting up a new RDMA connection, the first action by an RPC
     client will be to obtain a transport address for the server.  The
     mechanism used to obtain this address, and to open an RDMA
     connection is dependent on the type of RDMA transport, and outside
     the scope of this protocol.

So it seems perfectly reasonable and acceptable for the mapping of
NFS-RDMA onto IB to specify that the source and destination IP
addresses for an IB connection are placed in the CM private data.
This seems much easier than trying to turn ATS into an IETF standard.

 - R.