[openib-general] [IPoIB][RFC] neighbour/arp table manipulation

[James Lentini]

Both ip(8) and arp(8), the Linux system commands for managing the 
neighbour/arp table, have bugs dealing with IPoIB MAC addresses. 
Specifically they do not correctly handle the addition of new entries. 
For example, these commands will fail:

ip neigh add 192.168.0.138 lladdr 00:00:04:04:fe:80:00:00:00:00:00:00:00:01:73:00:00:00:8a:91 nud permanent dev ib0
arp -n -H infiniband -i ib0 -s 192.168.0.138 00:00:04:04:fe:80:00:00:00:00:00:00:00:01:73:00:00:00:8a:91

To the best of my knowledge this hasn't documented before.

In the case of the ip(8) command, the problem is simple. The command 
line parsing code requires the link layer address to be a maximum of 
16-bytes. Addresses over 16-bytes are truncated. This patch fixes the 
problem:

============================================
--- iproute2-2.6.15-060110/ip/ipneigh.c.orig	2006-03-10 16:08:37.361756000 -0500
+++ iproute2-2.6.15-060110/ip/ipneigh.c	2006-03-10 17:07:10.594429000 -0500
@@ -165,7 +165,7 @@ static int ipneigh_modify(int cmd, int f
 	addattr_l(&req.n, sizeof(req), NDA_DST, &dst.data, dst.bytelen);
 
 	if (lla && strcmp(lla, "null")) {
-		char llabuf[16];
+		char llabuf[20];
 		int l;
 
 		l = ll_addr_a2n(llabuf, sizeof(llabuf), lla);
============================================

I'm planning on sending this as a patch to netdev and the iproute2 
maintainer. Please let me know if you have any suggestions for 
improving the patch.

In the case of the arp(8) command, the situation is a bit more 
complicated. The arp command uses a SIOCSARP ioctl which takes a 
struct arpreq argument:

struct arpreq {
  struct sockaddr       arp_pa;      /* protocol address */
  struct sockaddr       arp_ha;      /* hardware address */
  int                   arp_flags;   /* flags            */
  struct sockaddr       arp_netmask; /* netmask (only for proxy arps) */
  char                  arp_dev[16];
};

where 

struct sockaddr {
  sa_family_t     sa_family;      /* address family, AF_xxx       */
  char            sa_data[14];    /* 14 bytes of protocol address */
};

The size of the IB HW address is 20 bytes (6 bytes greater than the 
space available in the struct sockaddr). The arp command doesn't check 
that the size of the user-specified hardware address will fit into a 
sockaddr. As a result, the IB HW address ends up spilling over into 
the arp_flags and, on 32-bit platforms, into the arp_netmask.

Between the arp command and the kernel's arp_req_set() function, the 
arp_flags field is set to ATF_PERM | ATF_COM = 0x6. This results in 
the 15th byte of the IB HW address always being set to 0x6.

To fix the problem, the arpreq structure needs to be updated and thus 
the userspace arp command and kernel ioctl code need to change.

I believe that the SIOCSARP ioctl is supported for backward 
compatibility and the arp command is deprecated. Therefore I don't 
believe this problem should be fixed (because it would break backward 
compatibility).