[openib-general] [RFC] IB address translation using ARP

[Michael Krause]

At 09:59 AM 10/12/2005, Caitlin Bestler wrote:
>
>
>
>----------
>From: openib-general-bounces at openib.org 
>[mailto:openib-general-bounces at openib.org] On Behalf Of Michael Krause
>Sent: Wednesday, October 12, 2005 8:24 AM
>To: Hal Rosenstock; Sean Hefty
>Cc: Openib
>Subject: RE: [openib-general] [RFC] IB address translation using ARP
>
>At 07:45 AM 10/10/2005, Hal Rosenstock wrote:
>>On Sun, 2005-10-09 at 10:19, Sean Hefty wrote:
>> > >I think iWARP can be on top of TCP or SCTP. But why wouldn't it care ?
>> >
>> > I'm referring to the case that iWarp is running over TCP.  I know that 
>> it can
>> > run over SCTP, but I'm not familiar with the details of that 
>> protocol.  With
>> > TCP, this is an end-to-end connection, so layering iWarp over it, only the
>> > endpoints need to deal with it.  I believe the same is true for SCTP.
>>
>>Yes, SCTP is similar in those regards.
>
>SCTP creates a connection and then multiplexes a set of sessions over 
>it.  You can conceptually think of it as akin to IB RD but where all QP 
>are bound to the same EEC.
>
>
>SCTP preserves all QP to QP semantics, including buffers posted to specific
>buffers and credits. So SCTP will allows multiple in-flight messages for each
>RDMA stream in the association.

Yep.  This is where iWARP differs from IB RD in that IB restricts this to a 
single in-flight message per EEC at a time while iWARP allows multiple 
in-flight over either transport type supported.  The logic behind why IB RD 
was constructed the way it was is somewhat complex but one of the core 
requirements was to enable a QP to communicate across multiple EEC while 
preserving an ordering domain within an EEC.  Given all of this needed to 
be implemented in hardware, i.e. without host software intervention, for 
both main data path and error management, the restriction to a single 
message was required.  I and several others had created a proprietary RDMA 
RC followed by a RD implementation 10+ years ago so we had a reasonable 
understanding of the error / complexity trade-offs.  Given the distances 
were within a usec or each other and one could support multiple EEC per 
endnode pair, the performance / scaling impacts were not seen as overly 
restrictive and met the software application usage models quite 
nicely.  Anyway, there are differences between iWARP / SCTP and IB RD so 
people cannot equate them beyond some base conceptual level aspects.

Mike 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20051012/3e9f6b21/attachment.html>