iWARP emulation protocol (was: [openib-general] RDMA connection and address translation API)

[Roland Dreier]

    Yaron> I send my proposal from 2004 re-send again as text
    Yaron> (attached) Also addresses the ServiceID issue, this can be
    Yaron> a baseline for discussions Feel free to change

I think this protocol is going in exactly the right direction.  Before
you sent this email, I had independently reached the conclusion that
what is desired is not a transport neutral API, but rather a general
protocol for emulating iWARP on IB.  Then it's easy to build an API
that covers both native iWARP and emulated iWARP on IB, and use that
for iSER and NFS/RDMA.

This has some nice properties.  For example, the high-level connection
API doesn't have to have a 64-bit service ID parameter any more -- we
can just pass in 16-bit TCP ports, and map them to IB service IDs.
Also, it's easy to put some filtering in the userspace CM to forbid
connections with source port < 1024 from unprivileged processes.  Then
listeners can have some level of trust in the source IP if the source
port is privileged.

I think that in light of the emerging consensus on using the IB CM
private data to carry IP address information, we can stop worrying
about ATS.  We can implement this private data mechanism immediately,
using a service ID base coming from the OpenIB OUI.  Once we have the
design nailed down, then we can go to the IBTA or IETF and standardize
a final service ID base.

I have a few minor quibbles with this proposal.  I think it would be
better to have only the IP version, source and destination IPs and
local in the CM private data.  The other fields don't seem generic to
all protocols.  If we do put the extra fields in the generic private
data, then we need an API to set them on active connect and get them
on passive connect, and I don't think it's worth it.

So I would suggest that there be no REP private data, and that the REQ
private data just be something like:

        0                   1                   2                   3     
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1   
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 00    |                       Src IP (127-96)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 04    |                       Src IP ( 95-64)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 08    |                       Src IP ( 63-32)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 12    |                       Src IP ( 31-00)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 16    |                       Dst IP (127-96)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 20    |                       Dst IP ( 95-64)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 24    |                       Dst IP ( 63-32)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 28    |                       Dst IP ( 31-00)                         |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  
 32    | IPVer |       Reserved        |        TCP Port               |  
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  

 - R.