[ofiwg] detecting FABRIC_DIRECT mismatch

[Hefty, Sean]

> > 	object->function_set->foobar(struct fi_foo *)
> 
> Okay, but in this case, if I follow correctly, the function that
> returns object->function_set is actually what has changed, ie it no
> longer returns the foobar pointer, so it should be name mangled and
> not present at all.

I'm not sure we're on the same page yet.  Here's a specific example:

Non-direct mode (from fi_endpoint.h)

static inline ssize_t
fi_recv(struct fid_ep *ep, void *buf, size_t len, void *desc,
	fi_addr_t src_addr, void *context)
{
	return ep->msg->recv(ep, buf, len, desc, src_addr, context);
}

Direct mode (from fi_direct_endpoint.h)

static inline ssize_t fi_recv(struct fid_ep *ep, void *buf, size_t len,
			      void *desc, fi_addr_t src_addr, void *context)
{
	return gnix_ep_recv(ep, buf, len, desc, src_addr, context);
}

gnix_ep_recv is added as an exported symbol in the direct GNI build.

One of the issues is that the context parameter may need to point to a specific data structure, which can differ between the direct and non-direct builds.  (It does not for the GNI provider, but does for B/G.)

Requiring that providers support both direct and non-direct clients simultaneously is one of the options being considered. 

- Sean