[ofw] winverbs ND provider

Sean Hefty sean.hefty at intel.com
Fri Jan 15 15:31:42 PST 2010


I traced down the main cause of the performance difference between the
winverbs and ibal ND providers.

The culprit is CWVDevice::FindPkey().  This converts a pkey to an index
and is called before transitioning the QP into the INIT state.  The ibal
provider hard codes the pkey index to 0, so lacks this problem.  After
modifying FindPkey() to return an index of 0, the connection rates were
1400 for ibal, versus 1300 for winverbs.  (Note that this affects the
performance of the librdmacm as well.)

Ideally, we want to support multiple pkeys, including handling dynamic
changes.  The current implementation handles this using a very simple
approach - always query for the pkey.  It shouldn't be too hard to
support multiple pkeys by caching the pkey table in userspace, but
handling dynamic changes is a little more difficult.

The design of winverbs is such that the query calls always drop into the
kernel to retrieve the latest data.  Caching was intended to be above
the interface, with the caller making use of CWVDevice::Notify() to know
when changes have occurred, so that a new query can be issued.

I haven't yet decided what to do about this yet, if anything.

- Sean




More information about the ofw mailing list