<html>
<body>
<font size=3>At 12:21 PM 1/7/2005, Libor Michalek wrote:<br>
<blockquote type=cite class=cite cite="">On Fri, Jan 07, 2005 at
11:03:47AM -0800, Josh England wrote:<br>
> This is probably not the right forum, but how would you say the
ES-API<br>
> compares with DAPL?<br><br>
ES-API appears to be much closer to the Linux AIO extensions then
to<br>
DAPL. DAPL provides an explicit RDMA API </font></blockquote><br>
DAPL and IT API (superset of DAPL) are designed to match RDMA hardware
semantics as closely as possible without requiring the application to be
coded to the verbs interface which may not be optimal for the typical
developer. Verbs interfaces like VAPI or the RNIC PI (a standard
verbs interface) are designed to be "too the metal" interfaces
which may not be reasonable for all but a subset of developers to
use.<br><br>
Sockets and the Extended Sockets API are designed to provide a general
purpose network API that can take advantage of existing and new
communication paradigms. The extensions were developed after close
evaluation of existing interfaces such as AIO, interface research, etc.
and working through their benefits, short-comings, etc. to find the right
balance in getting to the desired benefits. The extensions are able
to operate over a variety of network stack implementations enabling good
application portability / interoperability. Where an OS determines
that it can take advantage of RDMA, the extended Sockets provide
opportunities such as explicit memory management to make mapping to RDMA
as optimal as possible both from an implementation complexity perspective
as well as with minimal performance loss.<br><br>
<blockquote type=cite class=cite cite=""><font size=3>for applications
while AIO and ES-API allow for asynchronous operations on normal sockets,
so you can<br>
perform standard socket operations, plus asynchronous operations. AIO and
ES-API still maintain streaming socket symantics, while DAPL provides
full RDMA capabilities, such as explicit memory
placement.</font></blockquote><br>
Sockets / ES with SDP provides explicit memory placement but the
application developer does not have to worry about all of the details or
the interconnect-specifics. This has a number of advantages in
terms of broadening the application of RDMA interconnects to the larger
application space while still providing strong performance gains compared
to a standard network software implementation.<br><br>
<blockquote type=cite class=cite cite=""><font size=3>ES-API goes a few
steps further then the existing Linux AIO in providing explicit memory
registration, which may or may not be a significant win, depending on
application. Using Linux AIO, which does not have explicit memory
registration, my existing SDP implementation can reach full data rate.
The registration is done on the fly using FMRs</font></blockquote><br>
The cost of memory registration was evaluated in developing both iWARP
and the IB Verbs extensions. This was based on measurements taken
on IB and other hardware implementations. Not everyone was thrilled
with the performance cost thus we worked to get ES to support the
explicit memory management as well as provide more optimal memory
management verbs. I view this as an appropriate compromise since a
developer can make the choice on whether he / she manages the memory or
relies upon the underlying implementation do so on his / her behalf and
deal with whatever performance is actually delivered. There are
other benefits as well but are perhaps off topic from these
forums.<br><br>
Mike<br><br>
<br>
<blockquote type=cite class=cite cite=""><font size=3>-Libor<br><br>
<br>
> On Fri, 2005-01-07 at 07:12 -0800, Michael Krause wrote:<br>
> > At 03:57 PM 1/6/2005, Josh England wrote:<br>
> > > Is it the API that is completed, or is there an
implementation<br>
> > > written already?<br>
> > <br>
> > The API specification has been completed so now people can
implement<br>
> > if there is interest. The API is part of the Unix
branding effort<br>
> > done within the OpenGroup and is available to all for
free. Given the<br>
> > desire to implement open standards within the open source
community,<br>
> > this would seem like a logical API to support on Linux.
How this is<br>
> > actually started / implemented on Linux is an open
question. My main<br>
> > reason for providing the spec notification availability here is
that<br>
> > the socket extensions when combined with SDP will provide
optimum<br>
> > performance when used in conjunction with RDMA interconnects
while<br>
> > providing a fairly familiar interface to most network
application<br>
> > writers. For those that have implemented MPI over Sockets
(not all<br>
> > have done this), this would also provide a cleaner mapping and
still<br>
> > allow transparent access to RDMA with minimum performance
impact. <br>
> > <br>
> > Mike<br>
> > <br>
> > <br>
> > > -JE<br>
> > > <br>
> > > On Thu, 2005-01-06 at 10:31 -0800, Michael Krause
wrote:<br>
> > > > <br>
> > > > FYI.....The specification can be found at: <br>
> > > > <br>
> > > >
<a href="http://www.opengroup.org/bookstore/catalog/c050.htm" eudora="autourl">
http://www.opengroup.org/bookstore/catalog/c050.htm</a><br>
> > > > <br>
> > > > Use of this new interface will enable Sockets based
applications<br>
> > > to<br>
> > > > fully exploit the performance of RDMA interconnects
through the<br>
> > > SDP<br>
> > > > wire protocol. This API also provides
explicit memory management<br>
> > > > taking some of the guesswork out of this thorny
problem which can<br>
> > > > result in some performance loss and implementation
complexity<br>
> > > within<br>
> > > > the SDP layer.<br>
> > > > <br>
> > > > Mike <br>
> > > > <br>
> > > > <br>
> > > > <br>
> > > > <br>
> > > > <br>
> > > > Extended Sockets API (ES-API), Issue 1.0<br>
> > > > The Extended Sockets API (ES-API) Technical Standard
provides<br>
> > > > extensions to the traditional socket API to support
improved<br>
> > > > efficiency in network programming. The ES-API
includes:<br>
> > > synchronous IO<br>
> > > > and control operations on sockets; event queue-based
management of<br>
> > > > asynchronous operations; and pre-registering of
memory regions<br>
> > > that<br>
> > > > will be the subject of IO operations. These
facilities are<br>
> > > intended to<br>
> > > > support: improved efficiency when dealing with high
numbers of<br>
> > > socket<br>
> > > > file descriptors; 'zero-copy' transmit and receive
operations; and<br>
> > > > improved buffer management. The ES-API also includes
routines that<br>
> > > > provide asynchronous IO and control operations,
asynchronous<br>
> > > operation<br>
> > > > management, and memory registration functions for
applications<br>
> > > > manipulating sockets.<br>
> > > > <br>
> > > > <br>
> > > > <br>
> > > > Bibliographic Details<br>
> > > > Consortium Specifications <br>
> > > > <br>
> > > > Catalog number C050 <br>
> > > > ISBN 1931624526 <br>
> > > > Jan 2005<br>
> > > > <br>
> > > > OO. 72 pages. <br>
> > > > <br>
> > > > _______________________________________________<br>
> > > > openib-general mailing list<br>
> > > > openib-general@openib.org<br>
> > > >
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br>
> > > > <br>
> > > > To unsubscribe, please visit<br>
> > >
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br>
> <br>
> <br>
> _______________________________________________<br>
> openib-general mailing list<br>
> openib-general@openib.org<br>
>
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a><br>
> <br>
> To unsubscribe, please visit
<a href="http://openib.org/mailman/listinfo/openib-general" eudora="autourl">
http://openib.org/mailman/listinfo/openib-general</a>
</font></blockquote></body>
</html>