[openib-general] [RFC] IB address translation using ARP

[Michael Krause]

At 06:38 AM 9/30/2005, Caitlin Bestler wrote:
>
>
> > -----Original Message-----
> > From: openib-general-bounces at openib.org
> > [mailto:openib-general-bounces at openib.org] On Behalf Of Roland Dreier
> > Sent: Thursday, September 29, 2005 6:50 PM
> > To: Sean Hefty
> > Cc: Openib
> > Subject: Re: [openib-general] [RFC] IB address translation using ARP
> >
> >     Sean> Can you explain how RDMA works in this case?  This is simply
> >     Sean> performing IP routing, and not IB routing, correct?  Are you
> >     Sean> referring to a protocol running on top of IP or IB directly?
> >     Sean> Is the router establishing a second reliable connection on
> >     Sean> the backend?  Does it simply translate headers as packets
> >     Sean> pass through in this case?
> >
> > I think the usage model is the following: you have some magic
> > device that has an IB port on one side and "something else"
> > on the other side.  Think of something like a gateway that
> > talks SDP on the IB side and TCP/IP on the other side.
> >
> > You configure your IPoIB routing so that this magic device is
> > the next hop for talking to hosts on the IP network on the other side.
> >
> > Now someone tries to make an SDP connection to an IP address
> > on the other side of the magic device.  Routing tables + ARP
> > give it the GID of the IB port of this magic device.  It
> > connects to the magic device and run SDP to talk to the magic
> > device, and the magic device magically splices this into a
> > TCP connection to the real destination.
> >
> > Or the same idea for an NFS/RDMA <-> NFS/UDP gateway, etc.
> >
>
>Those examples are all basically application level gateways.
>As such they would have no transport or connection setup
>implications. The application level gateway simply offers
>a service on network X that it fulfills on network Y. But
>as far as network X is concerned the gateway IS the server.

It must be viewed as such.  The cross over point between the two domains 
represents independent management domains, trust domains, reliable delivery 
domains, etc.

>I do not believe it is possible to construct a transport
>layer gateway that bridges RDMA between IB and iWARP while
>appearing to be a normal RDMA endpoint on both networks.
>Higher level gateways will be possible for many
>applications, but I don't see how that relates to
>connection establishment. That would require having
>an end-to-end reliable connection, complete with flow
>control semantics, that bridged the two networks by
>some method other than encapsulation or tunneling.

We took steps to insure that both IB and iWARP could transmit packets in 
the main data path very efficiently between the two interconnects but it 
was never envisioned that a connection was truly end-to-end transparent 
across the gateway component.  I think most of the architects would not 
support such an effort to define such a beast.  There are many issues in 
attempting such an offering.  Just examine all of the problems with the 
existing iSCSI to FC solutions; they ignore a number of customer issues and 
hence have been relegated in many customer minds as TTM, play toys not 
ready for prime time.  This is one of the many reasons why iSCSI has not 
taken off as the hype portrayed.

It would be best to define a CM architecture that enabled communication 
between like endpoints and avoid the gateway dilemma. Let the gateway 
provider work out such issues as there are many requirements already on 
each side of these interconnects.

Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20051007/089d63d5/attachment.html>