[ofa-general] Re: 2.6.30.1: possible irq lock inversion dependency detected

[Roland Dreier]

 > Thanks for the patch. With the patch applied the lockdep warning
 > indeed occurs sooner and the output is now indeed shorter. You can
 > find the new lockdep output here:
 > http://bugzilla.kernel.org/attachment.cgi?id=22305.

Thanks, that actually looks like a completely different issue (that I
can actually understand).  I was able to reproduce that here: the issue
is doing skb_orphan() inside of priv->lock, and the network stack
locking is not irq-safe.  So the following hacky patch fixes that.

This would be a short-term solution for the immediate issue at least.  A
better solution would be if we didn't need to make priv->lock
hardirq-safe: the only place that requires it is the QP event handler in
ipoib_cm.c, and that might be a little dicy to fix.  Need to think about that.

However with this patch applied I don't see any further lockdep reports
here.  It would be great if you could retest yet again with this applied
(on top of my earlier patch to make priv->lock hardirq-safe as early as
possible).

Thanks,
  Roland

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index e319d91..4d53011 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -604,8 +604,11 @@ static void neigh_add_path(struct sk_buff *skb, struct net_device *dev)
 					   skb_queue_len(&neigh->queue));
 				goto err_drop;
 			}
-		} else
+		} else {
 			ipoib_send(dev, skb, path->ah, IPOIB_QPN(skb_dst(skb)->neighbour->ha));
+			spin_unlock_irqrestore(&priv->lock, flags);
+			return;
+		}
 	} else {
 		neigh->ah  = NULL;
 
@@ -689,6 +692,8 @@ static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev,
 			  be16_to_cpu(path->pathrec.dlid));
 
 		ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr));
+		spin_unlock_irqrestore(&priv->lock, flags);
+		return;
 	} else if ((path->query || !path_rec_start(dev, path)) &&
 		   skb_queue_len(&path->queue) < IPOIB_MAX_PATH_REC_QUEUE) {
 		/* put pseudoheader back on for next time */
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index a0e9753..a0825fe 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -720,7 +720,9 @@ out:
 			}
 		}
 
+		spin_unlock_irqrestore(&priv->lock, flags);
 		ipoib_send(dev, skb, mcast->ah, IB_MULTICAST_QPN);
+		return;
 	}
 
 unlock: