[openib-general] Re: openib-general Digest, Vol 22, Issue 114

[Roland Dreier]

    Bernard> On a multiple CPU system looking at TOP you see one
    Bernard> process consuming a full CPU. This happens to be the
    Bernard> thread handling completion queue entries.  I suggested
    Bernard> that we look at separate threads handing send completions
    Bernard> vs.  receive completions. When we ran with the split
    Bernard> completion queue patch, we no longer see one process
    Bernard> pegging the CPU at 100% and we get a speedup of 65% going
    Bernard> from STREAM to Duplex.

Right now completions are not handled in a thread -- they are handled
directly in the context of the CQ event interrupt handler.  And
splitting the CQ doesn't change the fact that there's only one
interrupt handler, which will only be running on one CPU at a time, so
I don't think this explanation holds up.

Changing things to run from thread context might be an improvement,
although the overhead of scheduling a thread might be a loss.  But I
could at least believe the explanation.  But I think we're still in
the dark about the patch here, since that's not what it does.

What would a good netperf command line be if I wanted to reproduce
your benchmark?  I have 4-way and 8-way machines available, so that's
no problem.

 - R.