[Ofvwg] Further thoughts on uAPI

[Jason Gunthorpe]

On Tue, Apr 26, 2016 at 10:19:37AM -0400, Doug Ledford wrote:
> For certain operations that have lots of optional items (work requests
> for one, work completions for another)

FWIW, I think we had a general consensus to take a different approach.

Basically, the 'common' uAPI does not care about micro-performance.

Drivers have to implement hardware-specific driver calls to micro-optimize
their own high speed paths, and that would be done specifically with a
single hardware in mind.

This is already done by the majority of drivers for wc/wr processing
(IIRC, only qib calls to the kernel for this)

If we do provide a common wr/wc API then it can just be designed
inefficiently around the netlink attribute architecture, uncaring
about performance because nothing should use it. I'd prefer not to
implement it at all...

This same basic idea flows over to other parts, eg if a driver has
special support for a specific work load (say fast creation of IB UD
AHs) then it can have a high speed driver-specific call to do that
work completely micro-optimized using data formed *exactly* the way
the hardware needs.

> base struct plus the length of all optional structs, and the order of
> the optional structs matches their bit order from lowest to highest in
> the magic element.  It's not quite as free form as the patches for
> timestamp support were, but still allows the structs some flexibility in
> what is included and what isn't.

Mellanox has a patch series that tries to do exactly this for the wc
in libibverbs - it is quite ugly, and the the benchmarks showed worse
performance compared to the current technique.

For the reasons above I would prefer to stick entirely with the
netlink attribute format or very similar as the main mechanism.

Jason