[openib-general] [RFC] IB address translation using ARP

[Sean Hefty]

>It is theoretically possible to support all this on an IPoIB based
>network. Multiple subnets, multiple routes to remote peers, ICMP
>redirect, multiple IP addresses for each physical interface, yada yada
>yada. But IMHO, the only way to do this would be to tie directly into
>the existing routing,  ARP, ICMP, etc... subsystems in Linux. Otherwise
>you'll end up recreating a gigantic (and I mean GIGANTIC) amount of

The current implementation ties into the standard Linux ARP tables.  If
connections were made over TCP/IP, using IPoIB, then I don't think that there
would be any issues.  The issues only arise because of the desire to use TCP/IP
network addresses over a non-TCP/IP network.

>code. This belief is why I've been a proponent of mapping GIDs to one
>and only one IP address and treating it for management purposes as the
>equivalent of an IP address. Without this, the whole mechanism for
>determining routes, etc.. breaks down. If you treat the GID like a MAC
>address -- it breaks, because a MAC address can have multiple IP
>addresses -- the observation that lead to the conclusion that ATS was
>broken in the first place.

We should be able to handle the case where a GID has multiple IP addresses bound
to it.  But even if we added a 1:1 restriction, the connection over IB issue
still exists.

>I know there is significant resistance to this idea, but I just don't
>see how we get this generically resolved without binding the two
>addressing schemes more closely. With the current binding, I just don't
>think it works.

Again, I don't think that the binding is the issue, so much as the desire to use
an address for a protocol that isn't actually being used for communication.  I
don't view a GID as an IP address because we're not sending and receiving IP
packets on the GID.  IPoIB treats GIDs as only part of a MAC address, which I
think is the proper view.

Anyway, returning back to the original problem of connecting to an IB gateway if
a given a destination IP address on a different subnet...  I'm slowly convincing
myself that either the CMA or AT should do this.  (I believe that the ib_addr
code will do this now, but still wasn't sure that we wanted it to.)

- Sean