[ofiwg] definition of a completion in OFI

[Hefty, Sean]

> What I read in your message is you have a connection oriented test,
> but no support for end of stream in the library, so I wouldn't ever
> expect that to work properly without races.

Okay, this is another option -- change the test.  :)

The test is polling for all completions, calling shutdown, then close.  It is not waiting for a shutdown event to be generated.

However, an app using an unconnected endpoint should have some reasonable expectation of things working.  Shutdown isn't usable here.

> Completion processing itself really has nothing to do with clean
> shutdown, and completions should only be expected to say the provider
> is done with the resource.

Then would you agree with applying my previous definition (slightly modified) for fi_shutdown to fi_close?

fi_close() - For reliable endpoints, blocks until all requests have been responded to by their respective destination(s), or the requests have timed out.  For unreliable endpoints, indicates that all requests have successfully been transmitted into the fabric.

My wording needs some help here.

> Not supporting clean shutdown is fine, it just means apps have to
> handle shutdown in-band, like IB already requires. Send an in-band
> shutdown message and when the far side echo's an ACK then things can
> be torn down. [And of course this is far more difficult with the new
> message ordering modes]

In-band shutdown seems like it would suffer from the same problem.  The sender of the ACK posts the send, then tears things down before the ACK is sent.

- Sean