[openib-general] [PATCH] 2/2 : ip over ib tx/rx split

[Michael S. Tsirkin]

Quoting r. Roland Dreier <roland at topspin.com>:
> Subject: Re: [openib-general] [PATCH] 2/2 : ip over ib tx/rx split
> 
>     Michael> I'm reposting this patch, its now tested. Together with
>     Michael> the first part: mthca send/receive q lock split, I am
>     Michael> getting about 2% more bandwidth with this patch.  I also
>     Michael> checked interrupt rate and it is about the same, which is
>     Michael> not surprising, since interrupts from both cqs are
>     Michael> coalesced by the eq polling in mthca.
> 
> Do you see any improvement with just the QP lock split and a single
> CQ?  It seems to me that the boost comes from being able to handle
> completions from one work queue while posting work requests to the
> other queue.

Didnt test that.

> I don't see how having two CQs helps reduce lock
> contention or improve parallelism at all.  There's still only one
> interrupt handler and one IPoIB completion event handler, which can
> only run on one CPU.  So it seems what will happen is:
> 
>     EQ event
>     Find EQE for receive CQ
>     Lock receive CQ, process completions
>     Unlock receive CQ
>     Find EQE for send CQ
>     Lock send CQ, process completions
>     Unlock send CQ
> 
> The only difference is that we unlock the receive CQ and lock the send
> CQ in the middle.

Are you sure?
I *do* have msi-x on these machines.

> 
> Especially since you say the interrupt rate is the same, it seems that
> almost all the time, the completion event handler is finding
> completions in both queues anyway.
> 
>  - R.
> 


-- 
MST - Michael S. Tsirkin