[ofw] RE: [PATCHv2] WinVerbs: Make QP modification asynchronous

[Fab Tillier]

> Can you re-run the test, but give the application one thread per core?

The app isn't multithreaded so I ran two apps side by side:

Async: 3562
Sync: 2705

The CPU wasn't pegged in either of these tests, so I ran again with 4 apps side by side...

Async: 3933
Sync: 3869

Still not CPU bound, but close and I think the part that isn't CPU bound is due to QP creation and destruction.  It's hard to get all 4 to start at the same time, too, so there's a little skew between them.

> I would also like to see the results of using the work item for an
> application that waits for the Modify to complete.  (This is how
> libibverbs, DAPL, or an IBAL compatibility layer would use the calls.  I
> don't think the ND provider uses the call.)

You mean strict serialization?  As in move each QP through its states, destroy it, and only then create the next?  Don't you have a test for that at the libibverbs level you can use?

> I would assume that adding this functionally to Connect() and Accept()
> would provide a result that's similar, but those are harder to test.

Yes, I have that coded but not yet tested.  The Connect/Accept change would help MSMPI since that's what the ND Provider does today.

Writing a test that establishes connections is significantly more complicated, though, and I wanted to demonstrate the benefits of the asynchronous operations and get buy in on that before really investing a lot of time/effort for something that's going to be rejected...