[Openframeworkwg] Some concerns about the new Fabric/IBverbs API

[Christoph Lameter]

The stripping out of branches etc would have to be tested. The compiler
needs to be able to figure out that the branches are not used. Otherwise
having more specialized function is useful regardless.

The indirection is still an issue because it increases the cache footprint.
The call tables tend to grow over time as functionality is added and so
there may be a continual slowdown. We have seen that in the kernel. If we
want to do the indirection tables then at least lets place all performance
sensitive calls in the same cache line.

Indirection are often a problem for prefetch and also an issue for the
compiler if global optimizations are desired.

The best result in terms of performance are obtained if the cache use for
code and data are minimal and if references are restricted to a minimum set
of Operating system pages as well as tlb entries. Having indirection tables
tends to lead to code and data being distributed here and there and thus
impacts latency and performance.



On Tue, Dec 10, 2013 at 1:53 PM, Hefty, Sean <sean.hefty at intel.com> wrote:

> > > > To do this a vendor just has to provide their own inlines for the hot
> > > > path function call tables, it isn't technically very challenging..
> > >
> > > Right. It would also allow the compiler to further optimize the code.
> >
> > Yes, and linking to the provider directly and use a relocation instead
> > of a table lookup with provide some additional tiny savings.
>
> Thinking about this more after the meeting, if we have a more generic
> call, such as ibv_post_send(), then there could be significant benefits to
> inlining the provider code.  The compiler should be able to strip out
> several branches and eliminate data structures.
>
> However, if we support optimized calls, such as
> rc_qp->post_rdma_write_with_immediate(), it would seem that the benefits of
> inlining would be minimal.  Ideally, there shouldn't be much that the
> compiler could do.
>
> - Sean
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20131210/a5282af8/attachment.html>