[Openframeworkwg] Some concerns about the new Fabric/IBverbs API

[Christoph Lameter]

On Fri, Dec 6, 2013 at 3:39 PM, Jason Gunthorpe <
jgunthorpe at obsidianresearch.com> wrote:

> On Fri, Dec 06, 2013 at 03:23:31PM -0800, Christoph Lameter wrote:
>
> > Too many ugly experiences with those. Skeleton files is something I
> > have found easy and usable with any editor of your picking. No
> > troubles with keeping a program functioning. A skeleton file can be
> > used as a reference and kept up to date. If a developer wants to
> > look something up he can do that with the skeleton
> > headers/sources. Or do a reference driver.
>
> The trick is avoiding massive code duplication. Provider functions
> need three possible variations:
>  - A static inline in a header
>  - A real function called through a linker dynamic relocation
>  - A real function called through the function pointer
>
> We'd have to find some kind of robust way to share all those identical
> implementations... The last two are obviously easy, the first is
> tricky, but maybe with some prepressor magic or something...
>
>
If we have common code then that could be put into some sort of low level
library. Code duplication is to some extend good because it allows easy
customization.



> > The normal libfabric path (fabric.h implicit include):
> >
> >      #ifdef LIBFABRIC_USE_MLX4_PROVIDER
> >      #include <fabric-mlx4-provider.h>
> >      #else
> >      static inline int indirect_func(OBJ *obj....) {return
> obj->function_table[..](obj,...);}
> >      #endif
> >
> > Maybe only provide one function and avoid the #ifdefs? The optional
> > abstraction layer can use that function and then its expanded if it
> > was inlined. I would like this to be as trivial as possible.
>
> There is no abstraction layer at this point - just the lowlevel
> provider functions. Either you call them through a function pointer
> mux or you arrange via compiler flags to have only one provider
> staticly inlined into your app.
>
>
I thought this was a proposal how it could be. The low level header files
by the provider can then specify only one implementation which can be
expanded elsewhere.


> I don't see how you'd have only one function when there are multiple
> providers and the generic function call mux that can supply an
> implementation.
>
>
Oh with dlopen you can load multiple implementation with the same function
names. The multiplex layer can build function vectors for these and provide
some sort of selection function for devices. Then we are back to the
existing overhead. The multiplex layer can supply many of the same
functions that the provider layer also offers and then route to the correct
driver that it may dynamically load.

This makes the whole scheme pretty simple. Either link and compile with the
headers of the provider library or with the multiplex ("abstraction
layer"). Multiplexing would replicate what the ibverbs layer does today and
similarly would not support the optimizations coming with inlining.
Additional functionality could be available at that multiplexing layer for
device discovery and selection.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20131206/54583269/attachment.html>