[ofw] RE: [PATCHv2] WinVerbs: Make QP modification asynchronous

[Fab Tillier]

Alright, I wrote a little test program that allocates QPs, moves them through the states INIT, RTR, RTS, ERROR, and destroys them.  It times how long it takes to do 10000 of these.

The program uses an I/O completion port and sits in a while loop.  Each time through the loop it will create a QP until it's reach the limit.  Each time through the loop it will poll the IOCP for a QP modify completion and if a completion is there it moves forward in the state machine, either requesting the next state change or releasing the QP if the QP is in error.  The test never blocks on the IOCP, it exclusively polls it (timeout of 0.)

Interesting observations:
If I leak the QPs instead of releasing them, the rate goes up from ~1800 to ~2200 (debug build of the drivers).  This makes sense because Release will synchronously destroy the QP, but illustrates the cost of synchronous operations.

Anyhow, here are the results, averages of 8 runs, on Windows Server 2008, x64, dual proc machines:

Sync WvModifyQp
Debug: 1798
Release: 1801

Async WvModifyQp
Debug: 2625
Release: 2634

The results are number of QPs create->INIT->RTS->RTR->ERROR->release per second.  So going with the work item gives almost 50% better performance.

-Fab