[Openframeworkwg] source code for proof of concept framework available

[Jason Gunthorpe]

On Tue, Dec 10, 2013 at 02:11:21AM +0000, Hefty, Sean wrote:

> > If obj->ops->future_function is not allowed to read unallocated memory
> > when linking to an old libfabric then it doesn't really matter if it
> > returns ENOSYS as it can never be called.
> 
> I was also concerned with providers that did not or could not
> implement certain functionality, and not just forward/backward
> compatibility.

It is an interesting area.. Returning a function pointer struct, and
returning null for some calls is a fairly robust and extensible way to
indicate provider support

On the other hand, should providers be allowed to half implement
functionality? What are apps going to do with that mess?

The best scenario for apps would be if the function pointer structs
were either fully implemented or not available at all. Then I can just
request the struct early on and bail if it isn't supported.

> The way I was thinking of it, there would have been 1 msg_ep per ep.
> But I think I see what you were thinking.  The difference seems to
> be whether you have:
> 
> msg_ep->send(msg_ep, ...)   or   msg_ep->send(ep, ...)
> 
> Assuming I'm following your thought correctly, this would provide an
> even smaller footprint per endpoint than what I have currently.

Yah, I was thinking of having a call to get the function pointer
struct, and that struct would not be allocated near every object.

It is a somewhat ugly API for applications, but it allows the most
optimization and provides some nice hooks to insert symbol versioning.

So apps would look like this:

  fi_qp *qp = fi_create_connected_qp("abc.net","service",....) 
  const fi_sendq_ops *sops = fi_get_sendq_ops(qp);
  sops->post_send(qp,...);
  sops->post_rdma_read(qp,..);
  [..]

Internally fi_get_sendq_ops returns a pointer to the single struct for
provider,sendq type - eg mlx4,rc_qp.

We can micro-optimize by doing this:

  const fi_senq_ops *fi_get_sendq_ops(fi_qp *qp) __attribute__((malloc))

Which tells the compiler that the result doesn't alias anything else
(aliasing rules more like vtable pointers in C++)

Maybe there is a way to make the get_*_ops cheap, and maybe some way to
be compatible with Christoph's inlining idea?

> I'll give this more thought, but I like the idea.  If you added some
> flags or attributes to the open_interface calls, you could even open
> the same interface multiple times, but with different optimizations
> defined, for a single endpoint.

I haven't looked at how the API works, just some of the details on how
the headers are setup and broad scale library design issues :)

-- 
Jason Gunthorpe <jgunthorpe at obsidianresearch.com>        (780)4406067x832
Chief Technology Officer, Obsidian Research Corp         Edmonton, Canada