[openib-general] RDMA connection and address translation API
rolandd at cisco.com
Wed Aug 24 14:03:26 PDT 2005
Tom> The issue is that this connection will be established when
Tom> the server may only want to accept requests that are
Tom> targetted to the 10.10.1.1 address. I don't get why this is
Tom> such a big deal. You can preclude this behavior by simply
Tom> keeping a one to one mapping between the IPv4 addresses and
Tom> the GIDs using the existing protocols and without mandating a
Tom> private data format across *all* ulps and transports.
Well, a few problems with what you say:
- ATS does not help at all with the case of a multi-homed interface.
Unless the remote system puts the IP it's trying to connect to
somewhere in the connection request, there is no way to be psychic
and recover this information.
- Mandating ATS use is dictating protocol design just as much as
requiring the CM private data to carry source and destination IP
- It's not just preventing connections to the wrong local address.
NFS-RDMA wants the remote source address (ie getpeername()) so that
it can look it up in the exports list.
- Saying that a given GID may only have a single IP address is
definitely a case of the cure being worse than the disease. I
don't think we can forbid perfectly valid multi-homed
configurations just because it's inconvenient for us to support them.
By the way, as far as I can tell, there is NO formal documentation of
the NFS-RDMA wire protocol. The current draft (draft-ietf-nfsv4-rpcrdma-01.txt)
This protocol is designed to function with equivalent semantics
over all appropriate RDMA transports. In its abstract form, this
protocol does not implement RDMA directly. [...] It therefore
becomes a useful, implementable standard when mapped onto a
specific RDMA transport, such as iWARP [RDDP] or Infiniband [IB].
In setting up a new RDMA connection, the first action by an RPC
client will be to obtain a transport address for the server. The
mechanism used to obtain this address, and to open an RDMA
connection is dependent on the type of RDMA transport, and outside
the scope of this protocol.
So it seems perfectly reasonable and acceptable for the mapping of
NFS-RDMA onto IB to specify that the source and destination IP
addresses for an IB connection are placed in the CM private data.
This seems much easier than trying to turn ATS into an IETF standard.
More information about the general