[openib-general] RE: [PATCH] [ib_addr] generalize address to RDMA device translation

[Caitlin Bestler]

Tom Tucker wrote:
> Sean:
> 
> This is a good start towards generalizing the IP address translation
> service. 
> 
> I think there is quite a bit of "hair" in this area that is
> "Part 2" of the iWARP integration -- after the integration of
> the current stuff into the trunk. I was going to introduce
> some of this "hair" at the face to face in Sonoma, but we
> should probably get it going now.
> 
> There are three areas that need attention:
> - ARP resolution
> - ROUTE changes
> - Path MTU changes
> 
> ARP Resolve
> 
> The iWARP side needs to be able to resolve an IP address to
> an Ethernet address. Today this is not done for iWARP and it
> works because the AMSO1100 does this itself in the hardware.
> Other iWARP devices probably don't. This means that the logic
> in ib_at needs to be extended on the iWARP side to call
> neigh_event_send (instead of arp_send) to resolve an IP to an
> Ethernet address.  The current method of calling arp_send
> directly and "sniffing" for arp replies is probably not the
> best way to go long term. It would be better to register for
> neighbor update events (new mechanism) and be notified when
> the neighbor entry gets resolved.
> This is better for two reasons: 1) it doesn't duplicate code
> already in Linux, and 2) unlike IB, Ethernet MAC addresses
> may change for the next hop while the connection is still
> active. The provider needs to know this so it's hardware ARP tables
> can be updated. 
> 

Agreed, an iWARP NIC *might* need to receive these notices,
so subscribing is an excellent approach.

Typically the iWARP RNIC is also a plain Ethernet NIC,
where ARP,etc. are being handled by the network stack.
It is a major problem if the RNIC and NIC are out of
sync on an L3 issue (like what the MAC address of a
local peer is, or what the next hop to reach a given
IP address is).

Sniffing is a workaround. Subscription is the correct
solution here for long term stability.

Subscription is also the only solution that will correctly
respond when the system administrator has manually loaded
an ARP entry.

This also applies to Routing and ICMP.