[openib-general] round 2 - proposal for socket based connectionmodel

[Kanevsky, Arkady]

Sean Hefty wrote:

> -----Original Message-----
> From: Sean Hefty [mailto:mshefty at ichips.intel.com] 
> Sent: Tuesday, October 25, 2005 6:44 PM
> To: Kanevsky, Arkady
> Cc: Sean Hefty; openib-general at openib.org; swg at infinibandta.org
> Subject: Re: [openib-general] round 2 - proposal for socket 
> based connectionmodel
> 
> 
> Kanevsky, Arkady wrote:
> > What are you trying to achieve?
> 
> I'm trying to define a connection *service* for Infiniband 
> that uses TCP/IP 
> addresses as its user interface.  That service will have its 
> own protocol, in 
> much the same way that SDP, SRP, etc. do today.
> 
> > I am trying to define an IB REQ protocol extension that support IP 
> > connection 5-tuple exchange between connection requestor and 
> > responder.
> 
> Why?  What need is there for a protocol extension to the IB 
> CM?  To me, this is 
> similar to setting a bit in the CM REQ to indicate that the 
> private data format 
> looks like SDP's private data.  The format of the _private_ 
> data shouldn't be 
> known to the CM; that's why it's private data.

There is no requirement that the remote side uses the same Linux CM.
So in order to achieve interopability you need a protocol.
SDP hello-world protocol is defined for SDP.
We are defining an equivalent that is ULP independent.

If CM is not involved then it is ULP that populate the 5-tuple
info on requestor side and parses it on the remote side.
Thus, make ULP CM IB specific.
This is what we are trying to avoid.
ULP should not change regardless whether or not it is running
on IB, iWARP, VIA or any other RDMA transport.

iWARP does not need private data to pass 5-tuple.

> 
> > And define mapping between IP 5-tuple and IB entities.
> 
> No mapping between IP <-> IB addresses was defined in the 
> proposal.  Defining 
> this mapping is required to make this work.  Right now, the 
> mapping is the 
> responsibility of every user.
> 
> > That way ULP which was written to TCP/IP, UDP/IP, CSTP/IP 
> (and so on) 
> > can use RDMA transport without change.
> 
> A ULP written to TCP/IP can use an RDMA transport without 
> change.  They use SDP. 
>   However, an application that wants to take advantage of QP 
> semantics must 
> change.  (And if they want to take full advantage of RDMA, 
> they'll likely need 
> to be re-architected as well.)  The goal in that case becomes 
> to permit them to 
> establish connections using TCP/IP addresses.
> 
> To meet this goal, we need to define how to map IP address to 
> and from IB 
> addresses.  That mapping is part of the protocol, and is 
> missing from the 
> proposal.  And if the application isn't going to know that 
> they're running on 
> Infiniband, then the mapping must also include mapping to a 
> destination service ID.
> 
> > To modify ULP to know that it runs on top of IB vs. iWARP
> > vs. (any other RDMA transport) is bad idea.
> > It is one thing to choose proper port to connect.
> > Completely different to ask ULP to parse private data
> > in transport specific way.
> > The same protocol must support both user level ULPs
> > and kernel level ULPs.
> 
> Defining an interface that allows a ULP to use either iWarp, 
> IB, or some other 
> random RDMA transport is an implementation issue.  However, 
> it requires 
> something that maps IP to IB addresses (including service IDs).
> 
> To be more concrete, you've gone from having source and 
> destination TCP/IP 
> addresses to including them in a CM REQ.  What translated the 
> source and 
> destination IP addresses into GIDs and a PKey?  Who converted 
> those into IB 
> routing information?  How was the destination of the CM REQ 
> determined?  What 
> service ID was selected?

IPoIB defines IP -> GID
Port -> IB Service ID (part of this proposal)
Pkey is configuration setup done by administrator.
Ditto for VLAN.

> 
> - Sean
>