[ofa-general] Re: 2.6.30.1: possible irq lock inversion dependency detected

[Roland Dreier]

 > >And that will deadlock waiting in del_timer_sync() since the timer has
 > >been interrupted by an interrupt handler that will spin on a spinlock
 > >that is part of this chain.
 > >
 > >I'm not sure what the right fix is.  It does seem to me that this should
 > >be fixed within the ib_mad module, since doing del_timer_sync() within a
 > >spinlocked region seems like the fundamental problem.  However I'm not
 > >sure what the best way to rewrite the ib_mad usage is.
 > 
 > If I followed this correctly, will moving calls to cancel_delayed_work() outside
 > of any spinlocks fix this?  (If so, it's not immediately obvious to me what the
 > best fix is either.)

Yes, I think that if cancel_delayed_work() and hence del_timer_sync() is
outside of any other locks then there is no deadlock -- you can think of
del_timer_sync() as being like a lock (which is how lockdep tracks it too).
But of course we can't really do that because that leaves the timeout
tracking unlocked and racy in the mad module.

The best idea I can come up with so far is to move to an explicit timer
in the mad module, so that we can do mod_timer() inside the lock rather
than having to do the equivalent of del_timer_sync() + add_timer()
(implicitly through the delayed work API).  But that unfortunately is
somewhat invasive surgery for the mad module... definitely doable but
ideally there would be an easier way.

I guess we could add a "requeue_delayed_work()" API to the kernel
workqueue stuff that does mod_timer() instead of adding it, but it might
be tricky to get the interface to that right.

 - R.