[Rdma-developers] Re: [openib-general] OpenIB and OpenRDMA: Convergence on common RDMA APIs and ULPs for Linux

[Michael Krause]

At 09:29 AM 5/27/2005, Grant Grundler wrote:
>On Fri, May 27, 2005 at 07:24:44AM -0700, Michael Krause wrote:
>...
> > Again, Sockets is an application API and not how one communicates to a TOE
> > or RDMA component.
>
>Mike,
>What address family is used to open a socket over iWARP? AF_INET?
>Or something else?

TCP = AF_INET.  Address family != Sockets.  Sockets is an API that can 
operate over multiple address families.  An application can be coded to 
Sockets, IT API, DAPL, or a verbs interface like the RNIC PI.  It is a 
matter of choice as well as what is trying to be accomplished.  The RNIC PI 
is an acceptable interface for any RDMA-focused ULP.  There are pros / cons 
to using such a verbs interface directly but I do not believe any one can 
deny that a general-purpose verbs API is a good thing at the end of the day 
as it works for the volume verbs definition.  Whether one applies further 
hardware semantics abstraction such as IT API / DAPL should be a choice for 
the individual subsystem as there is no single right answer across all 
subsystems.  Attempting to force fit isn't practical.

>I understand most of what you wrote but am still missing one bit:
>How is the RNIC told what the peer IP is it should communicate with?

The destination address (IB GID or IP) is derived from the CM 
services.  This is where the two interconnects differ in what is required 
to physical inject a packet on the wire.  This is why I call it out as 
separate from the verbs interface and something that could be abstracted to 
some extent but at the end of the day, really requires the subsystem to 
understand the underlying fabric type to make some intelligent 
choices.  Given this effort is still nascent, most of the issues beyond 
basic bootstrap have not really been discussed as yet.



> > The RNIC PI has been proposed as an interface to the
> > RDMA functionality.  The PI supports all of the iWARP and IB v 1.2 verbs.
>
>That's good. Folks from RDMA consortium will have to look at openib 
>implementations and see whats missing/wrong.
>Then submit proposals to fill in the gaps. I'm obviously not the first one 
>to say this.

There are two open source efforts.  The question is whether to move to a 
single effort (I tried to get this to occur before OpenIB was formally 
launched but it seem to fall on deaf ears for TTM marketing purposes) or 
whether to just coordinate on some of the basics.  My preference remains 
that the efforts remained strictly focused on the RDMA infrastructure and 
interconnect-specific components and leave the ULP / services as separate 
efforts who will make their own decisions on how best to interface with the 
RDMA infrastructure.

>I expect most of the principals involved with openib.org do NOT have time 
>to browse through RNIC PI at this point. They are struggling to get 
>openib.org filled in sufficiently so it can go into a commercial distro 
>(RH/SuSE primarily).

Hence, why OpenRDMA needs to get source being developed to enable the RNIC 
community.  If people find value in the work, then people can look at 
finding the right solution for both IB and iWARP when it makes sense.


>Revenue for them comes from selling IB equipment. Having openib.org code 
>in kernel.org is a key enabler for getting
>into commercial distros.  I expect the same is true for RNIC vendors as well.
>
>RNIC Vendors (and related switch Vendors) will have to decide which path 
>is the right one for them to get the support into kernel.org. Several 
>openib.org people have suggested one (like I have). RNIC folks need to 
>listen and decide if the advice is good or not. If RNIC folks think they 
>know better, then please take another look at where openib.org is today 
>and where rdmaconsortium is.
>
>I'm certain openib.org would be dead now if policies and direction changes 
>had not made last year as demanded by several key linux developers and 
>"users" (Gov Labs).

Understood.

Mike 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050527/19cd5aee/attachment.html>