[ofa-general] [PATCH v2] ipiob: fix rtnl deadlock

[Roland Dreier]

OK, how about the patch below?  The idea is that we don't really care
about tasks that happen when running ipoib_stop(), except we don't want
netif_carrier_on() to race with bringing down the interface -- but I
think adding rtnl_lock() around that is sufficient.

Oh and the calls to flush_scheduled_work() were bogus, since we never
use schedule_work() in the first place.  It only makes sense to flush
the real ipoib workqueue.

This seems to work fine on my test system and produces no lockdep
warnings... any testing and/or review would be good, but I'm pretty
happy with this approach.

 - R.

 drivers/infiniband/ulp/ipoib/ipoib_main.c      |   19 +++++++++----------
 drivers/infiniband/ulp/ipoib/ipoib_multicast.c |   10 +++++++++-
 2 files changed, 18 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index f51201b..7e9e218 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -156,14 +156,8 @@ static int ipoib_stop(struct net_device *dev)
 
 	netif_stop_queue(dev);
 
-	/*
-	 * Now flush workqueue to make sure a scheduled task doesn't
-	 * bring our internal state back up.
-	 */
-	flush_workqueue(ipoib_workqueue);
-
-	ipoib_ib_dev_down(dev, 1);
-	ipoib_ib_dev_stop(dev, 1);
+	ipoib_ib_dev_down(dev, 0);
+	ipoib_ib_dev_stop(dev, 0);
 
 	if (!test_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags)) {
 		struct ipoib_dev_priv *cpriv;
@@ -1314,7 +1308,7 @@ sysfs_failed:
 
 register_failed:
 	ib_unregister_event_handler(&priv->event_handler);
-	flush_scheduled_work();
+	flush_workqueue(ipoib_workqueue);
 
 event_failed:
 	ipoib_dev_cleanup(priv->dev);
@@ -1373,7 +1367,12 @@ static void ipoib_remove_one(struct ib_device *device)
 
 	list_for_each_entry_safe(priv, tmp, dev_list, list) {
 		ib_unregister_event_handler(&priv->event_handler);
-		flush_scheduled_work();
+
+		rtnl_lock();
+		dev_change_flags(priv->dev, priv->dev->flags & ~IFF_UP);
+		rtnl_unlock();
+
+		flush_workqueue(ipoib_workqueue);
 
 		unregister_netdev(priv->dev);
 		ipoib_dev_cleanup(priv->dev);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index 8950e95..ac33c8f 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -392,8 +392,16 @@ static int ipoib_mcast_join_complete(int status,
 					   &priv->mcast_task, 0);
 		mutex_unlock(&mcast_mutex);
 
-		if (mcast == priv->broadcast)
+		if (mcast == priv->broadcast) {
+			/*
+			 * Take RTNL lock here to avoid racing with
+			 * ipoib_stop() and turning the carrier back
+			 * on while a device is being removed.
+			 */
+			rtnl_lock();
 			netif_carrier_on(dev);
+			rtnl_unlock();
+		}
 
 		return 0;
 	}