<html>
<body>
<font size=3>At 09:59 AM 10/12/2005, Caitlin Bestler wrote:<br>
<blockquote type=cite class=cite cite=""> <br><br>
<dl><hr>
</font>
<dd><font face="tahoma" size=2>From:</b>
openib-general-bounces@openib.org
[<a href="mailto:openib-general-bounces@openib.org" eudora="autourl">
mailto:openib-general-bounces@openib.org</a>] On Behalf Of </b>Michael
Krause<br>
<dd>Sent:</b> Wednesday, October 12, 2005 8:24 AM<br>
<dd>To:</b> Hal Rosenstock; Sean Hefty<br>
<dd>Cc:</b> Openib<br>
<dd>Subject:</b> RE: [openib-general] [RFC] IB address translation using
ARP<br>
</font><font size=3><br>
<dd>At 07:45 AM 10/10/2005, Hal Rosenstock wrote:<br>
<blockquote type=cite class=cite cite="">
<dd>On Sun, 2005-10-09 at 10:19, Sean Hefty wrote: <br>
<dd>> >I think iWARP can be on top of TCP or SCTP. But why wouldn't
it care ?<br>
<dd>> <br>
<dd>> I'm referring to the case that iWarp is running over TCP.
I know that it can<br>
<dd>> run over SCTP, but I'm not familiar with the details of that
protocol. With<br>
<dd>> TCP, this is an end-to-end connection, so layering iWarp over
it, only the<br>
<dd>> endpoints need to deal with it. I believe the same is true
for SCTP.<br><br>
<dd>Yes, SCTP is similar in those regards.</blockquote><br>
<dd>SCTP creates a connection and then multiplexes a set of sessions over
it. You can conceptually think of it as akin to IB RD but where all
QP are bound to the same EEC.<br><br>
<br>
</font>
</dl><font face="arial" size=2 color="#0000FF">SCTP preserves all QP to
QP semantics, including buffers posted to specific<br>
buffers and credits. So SCTP will allows multiple in-flight messages for
each<br>
RDMA stream in the association.</font></blockquote><br>
Yep. This is where iWARP differs from IB RD in that IB restricts
this to a single in-flight message per EEC at a time while iWARP allows
multiple in-flight over either transport type supported. The logic
behind why IB RD was constructed the way it was is somewhat complex but
one of the core requirements was to enable a QP to communicate across
multiple EEC while preserving an ordering domain within an EEC.
Given all of this needed to be implemented in hardware, i.e. without host
software intervention, for both main data path and error management, the
restriction to a single message was required. I and several others
had created a proprietary RDMA RC followed by a RD implementation 10+
years ago so we had a reasonable understanding of the error / complexity
trade-offs. Given the distances were within a usec or each other
and one could support multiple EEC per endnode pair, the performance /
scaling impacts were not seen as overly restrictive and met the software
application usage models quite nicely. Anyway, there are
differences between iWARP / SCTP and IB RD so people cannot equate them
beyond some base conceptual level aspects.<br><br>
Mike</body>
</html>