[ofa-general] Race condition in core/sysfs.c (kernel panic) when unloading the driver

[Roland Dreier]

 > We have found a race condition in sysfs.c which occurs when unloading low-level modules
 > (e.g., mlx4_ib) in the driver.
 > Specifically:
 > 
 > Although the kernel takes reference counts on sysfs files, it does not take such counts
 > on modules which implement attribute reads.
 > 
 > For example, we have:
 > static ssize_t show_port_pkey(struct ib_port *p, struct port_attribute *attr,
 > 			      char *buf)
 > {
 > 	struct port_table_attribute *tab_attr =
 > 		container_of(attr, struct port_table_attribute, attr);
 > 	u16 pkey;
 > 	ssize_t ret;
 > ====>race condition HERE *****
 > 	ret = ib_query_pkey(p->ibdev, p->port_num, tab_attr->index, &pkey);
 > 	if (ret)
 > 		return ret;
 > 
 > 	return sprintf(buf, "0x%04x\n", pkey);
 > }

I've not been able to reproduce this on a current kernel.  I tried
adding the patch below to make the race window very big:

--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -273,6 +273,9 @@ static ssize_t show_port_pkey(struct ib_port *p, struct port_attribute *attr,
 	u16 pkey;
 	ssize_t ret;
 
+	printk(KERN_ERR "enter show_port_pkey\n");
+	msleep(10000);
+	printk(KERN_ERR "call ib_query_pkey\n");
 	ret = ib_query_pkey(p->ibdev, p->port_num, tab_attr->index, &pkey);
 	if (ret)
 		return ret;

so show_port_pkey() waits 10 seconds before actually calling
ib_query_pkey().  Then I do something like

    cat /sys/class/infiniband/mlx4_0/ports/1/pkeys/0

in one shell and immediately (during the 10 second window before
ib_query_pkey() is called):

    modprobe -r mlx4_ib

in another shell.  And I see that the mlx4_ib module is not removed
until the read of the pkey file completes; this is as I would expect,
since the sysfs delete of the pkey file should wait until there are no
open fds for that file.

What test are you using to hit this race?  Are you using a distro kernel
with OFED?

 - R.