[openib-general] [PATCH] 1/2 separate locking for send and receive q in mthca

[Michael S. Tsirkin]

Quoting r. Roland Dreier <roland at topspin.com>:
> Subject: Re: [openib-general] [PATCH] 1/2 separate locking for send and receive q in mthca
> 
> I finally got around to reviewing this, and I think I see a problem.
> It seems that if we get, say, a send completion followed by a receive
> completion, we will not update which wq is locked so we'll update the
> receive queue with the send queue locked.  Is this true?
> 
>  - R.
> 


Ugh. True. No idea what was I thinking, or why is it working so well.
To fix this, I'd have to store the is_send flag between runs.

Instead of doing this, how about something else I would prefer: we could
avoid locking the QP on CQ poll altogether, if there is a separate
last polled index that is written only by cq poll and read by
qp post.
This index update would be protected by a cq lock.

We thus have a circular buffer along the lines described in LDD2
(http://www.xml.com/ldd/chapter/book/ch09.html#t8).

We could still get "false positives" where post sees the qp is
full since polled index (consumer) is updated by another cpu.
To solve this, if we see a full qp, we lock the cq and re-test.
This is a rare case so it should not hurt.

If all this makes sence to you I'll send a patch.

-- 
MST - Michael S. Tsirkin