[openib-general] round 2 - proposal for socket based connectionmodel

[Sean Hefty]

Kanevsky, Arkady wrote:
> What are you trying to achieve?

I'm trying to define a connection *service* for Infiniband that uses TCP/IP 
addresses as its user interface.  That service will have its own protocol, in 
much the same way that SDP, SRP, etc. do today.

> I am trying to define an IB REQ protocol extension that
> support IP connection 5-tuple exchange between connection
> requestor and responder.

Why?  What need is there for a protocol extension to the IB CM?  To me, this is 
similar to setting a bit in the CM REQ to indicate that the private data format 
looks like SDP's private data.  The format of the _private_ data shouldn't be 
known to the CM; that's why it's private data.

> And define mapping between IP 5-tuple and IB entities.

No mapping between IP <-> IB addresses was defined in the proposal.  Defining 
this mapping is required to make this work.  Right now, the mapping is the 
responsibility of every user.

> That way ULP which was written to TCP/IP, UDP/IP, CSTP/IP (and so on)
> can use RDMA transport without change.

A ULP written to TCP/IP can use an RDMA transport without change.  They use SDP. 
  However, an application that wants to take advantage of QP semantics must 
change.  (And if they want to take full advantage of RDMA, they'll likely need 
to be re-architected as well.)  The goal in that case becomes to permit them to 
establish connections using TCP/IP addresses.

To meet this goal, we need to define how to map IP address to and from IB 
addresses.  That mapping is part of the protocol, and is missing from the 
proposal.  And if the application isn't going to know that they're running on 
Infiniband, then the mapping must also include mapping to a destination service ID.

> To modify ULP to know that it runs on top of IB vs. iWARP
> vs. (any other RDMA transport) is bad idea.
> It is one thing to choose proper port to connect.
> Completely different to ask ULP to parse private data
> in transport specific way.
> The same protocol must support both user level ULPs
> and kernel level ULPs.

Defining an interface that allows a ULP to use either iWarp, IB, or some other 
random RDMA transport is an implementation issue.  However, it requires 
something that maps IP to IB addresses (including service IDs).

To be more concrete, you've gone from having source and destination TCP/IP 
addresses to including them in a CM REQ.  What translated the source and 
destination IP addresses into GIDs and a PKey?  Who converted those into IB 
routing information?  How was the destination of the CM REQ determined?  What 
service ID was selected?

- Sean