[openfabrics-ewg] [openib-general] OFED 1.1 release - schedule and features

[Michael Krause]

At 03:49 PM 7/12/2006, Fabian Tillier wrote:
>Hi Mike,
>
>On 7/12/06, Michael Krause <krause at cup.hp.com> wrote:
>>
>>At 09:48 AM 7/12/2006, Jeff Broughton wrote:
>>
>>>Modifying the sockets API is just defining yet another RDMA API, and we have
>>>so many already....
>>
>>I disagree.  This effort has distilled the API to basically one for RDMA
>>developers.  Applications are supported over this via either MPI or Sockets.
>
>There's been a lot of effort to make the RDMA verbs easy to use.  With
>the RDMA CM, socket-like connection semantics can be used to establish
>the connection between QPs.  The connection establishment is the hard
>part - doing I/O is trivial in comparisson.  This verbs and RDMA CM
>have nothing to do with MPI.
>
>If an application is going to be RDMA aware, I don't see any reason it
>shouldn't just use the verbs directly and use the RDMA CM to establish
>the connections.

What's your point?  It seems you are in agreement that there is a single 
RDMA API that people can use.


>>    It seems rather self limiting to think the traditional BSD synchronous
>>Sockets API is all the world should be able to use when it comes to Sockets.
>>  Sockets developers could easily incorporate the extensions into their
>>applications providing them with improved designs and flexibility without
>>having to learn about RDMA itself.
>
>Wait, you want applications to be able to register memory and issue
>RDMA operations, but not have to learn about RDMA?  How does that make
>sense?

The Sockets API extensions allow developers to register memory.   That has 
been a desire by many when it comes to SDP or copy avoidance technology as 
it optimizes the performance path by eliminating the need to do per op 
registration.  For many applications which already known working sets, they 
can use this to enable the OS and underlying infrastructure to take 
advantage of this fact to improve performance and quality of the 
solution.  The extensions provide the async communications and event 
collection mechanisms to also improve performance over the rather limiting 
select / poll supported by Sockets today.

It currently does not support explicit RDMA but it is rather trivial to add 
such calls and remove the need to interject SDP if desired.    The benefits 
of such new API extensions are there for those that want to eliminate one 
more ULP with its unfortunate IP cloud over head.


>>  If the couple of calls necessary to
>>extend this API to support direct RDMA would allow them to eliminate SDP
>>entirely, well, that has benefits that go beyond just its all Sockets;
>
>For a socket implementation to support RDMA, the socket must have an
>underlying RDMA QP.  This means that if you want the application to
>not have to be verbs-aware, you can't really get rid of SDP - you're
>just extending SDP to let the application have a part in memory
>registration and RDMA, while still supporting the traditional BSD
>operations.  This is IMO more complex than just letting applications
>interface directly with verbs, especially since the SDP implementation
>will size the QP for its own use, without a means for negotiating with
>the user so that you don't cause buffer overruns.

Please take a look at the API extensions.   I never stated that one gets 
rid of SDP unless one adds the RDMA-explicit calls.

As for complexity, well, the goal is to extend to Sockets developers the 
optimal communication paradigm already available on OS such as Windows 
without having to leave with the same unfortunate constraints imposed by 
the OS.  The same logic applies to extending the benefits derived from MPI 
which supports async communications as well as put / get semantics which 
would be analogous to the additional RDMA interfaces I referenced.

I find it strange that people would argue against improving the Sockets 
developer's tool suite when the benefits are already proven elsewhere 
within the industry and even within this open source effort.  Giving the 
millions of Sockets developers the choice of a set of extensions that work 
over both RDMA and traditional network stacks seems like a no 
brainer.  Trying to force them to use a native RDMA API even if 
semantically similar to Sockets seems like a poor path to pursue.  Leave 
the RDMA API to the middleware providers and those that need to be close 
the metal.


>>it also eliminates the IP cloud that hovers over SDP licensing.   Something
>>that many developers and customers would appreciate.
>
>I believe that Microsoft's IP claims only apply to SDP over IB -- I
>don't believe SDP over iWarp is affected.  I don't know how the RDMA
>verbs moving towards a hardware independent (wrt IB vs. iWarp) affects
>the IP claims, but it should certainly make things interesting if a
>single SDP code base can work over both IB and iWarp.

SDP is SDP and it isn't just restricted to IB.   I'll leave it to the 
lawyers to sort it out but having a single SDP with minor code execution 
path deltas for the IB-specifics isn't that hard to construct.  It has been 
done on other OS already.

Mike

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20060713/68d200ba/attachment.html>