[openib-general] Re: [PATCH] cm refcount race fix

[Sean Hefty]

Roland Dreier wrote:
>     Sean> Could we use atomic_dec_and_lock() instead?  This would keep
>     Sean> refcount atomic, but use a spinlock to synchronize with
>     Sean> destruction.
> 
> Hmm, how does that help?
> 
> Just going to a plain integer with a spinlock to protect it seems
> simple and clear.

Basically, I'm just trying to explore what options we have.

The cost of using a spinlock around an integer is that we end up serializing 
everything with the larger lock.  With the CM, sometimes the global CM lock is 
being held when refcount is incremented, but there are places where only a lock 
on the cm_id is held.  And unless the id is being destroyed, there's no need to 
acquire the lock.

Thinking about this more, it seems that we're wanting something similar to:

initialize()
{
	get destroy mutex
}

release()
{
	if (atomic_dec_and_test(refcount))
		put destroy mutex;	/* or signal event */
}

destroy()
{
	release()
	get destroy mutex;		/* wait for event to be signaled */
}

Using an actual mutex gets ugly since it's held for a long time, and ends up 
needing to be released in destroy().  And I don't see that there's an event 
abstraction that would work.

- Sean