[swg] Re: [openib-general] TCP/IP connection service over IB

[Tom Tucker]

On Tue, 2005-10-25 at 13:16 -0700, Ted H. Kim wrote:
> Tom,
> 
> Some comments inline ...
> 
> 
> Tom Tucker wrote:
> > I think it's relevant, so let's make sure my assumptions are correct:
> > 
> > - The ITAPI will be a "ULP" on OpenIB
> 
> ITAPI is like uDAPL, so if uDAPL is a "ULP" then the answer is yes.
> The point is that for uDAPL you have the actual "app" running over
> uDAPL. So I guess it's a matter of terminology whether uDAPL is
> a ULP or is it some sort of middleware with the app being the "ULP".
> 

Yeah, you're right the terminology is probably a little goofy. The
reason for the goofosity is that some of the "ulp" really are protocols
(ISER, IPoIB), and some are API (DAPL, MPI). All use the same interface 
to register with OpenIB. 

But that said, yes, ITAPI is like uDAPL.

> 
> > - The ITAPI will create the IRD/ORD headers in its private data and
> > submit this as part of its connection establishment. 
> > - The ITAPI consumer at the remote peer will use this data to configure
> > it's local QP before accepting the connection
> > 
> > Over IB, the IRD/ORD private data will be prepended with a "private data
> > header" that contains the source and destination IP addresses, source
> > port, etc... The remote peer will not see this data as part of the
> > private data, but rather will see it in the CMA event in the upcall.
> 
> Over IB, the IRD/ORD data is already built in to the standard CM
> stuff (i.e. the "responder resources" and "initiator depth" fields of
> REQ and REP). So no additional demands are made on private data for IB
> in ITAPI for the IOH purpose. Of course the ITAPI app (like a uDAPL app)
> can also use private data for app specific/ULP reasons.

ok -- bad example. Sorry. This is a weird one. On iWARP, you need the
private data header to pass this stuff along and on IB, you don't. What
I was trying to say is that "whatever the private data", on IB it will
get a private data header prepended and on iWARP, it won't.

> 
> 
> > Over iWARP/MPA, there will be nothing else in the private data except
> > what was provided by the consumer (ITAPI in this case). The reason being
> > that this extra information (IP addressing info) is in the protocol
> > header proper.
> 
> Just to restate for clarity, ITAPI for iWARP will use the first 16 bytes of
> MPA private date for the IOH (IRD/ORD header). The rest is usable for
> app/ULP reasons.

Yessir. And in fact, the ITAPI CM will strip this stuff before
presenting it to the app.

> 
> 
> I should point out that there was once a proposal of doing a RDDP IETF
> draft which would have sub-divided the MPA private data into a
> "middleware" section and an "app" section. The idea was to be sure that
> the app/ULP and middleware (e.g. the IOH) uses of private data would not
> step on each other. I think this idea did not progress, mostly because
> the author (John Carrier, formerly of Adaptec) changed jobs and was no
> longer working on iWARP stuff.
> 
> While not directly proposed, this idea could have been carried over to IB.
> Some of the ideas on this thread are already implicitly
> doing this middleware (for IP addressing purpose) vs ULP/app split.
> 

I think we are grappling with a lot of these layering issues now. We are
also grappling with protocol vs. implementation issues.  

Keep it coming, because this is exactly the kind of feedback I think we
need.

> -ted
>