[ofiwg] proposal to add a new completion semantic

[Hefty, Sean]

Based on discussion with some MPI developers, I'm proposing that we introduce a new completion semantic to libfabric.  This specifically targets support for the MPI synchronous send operation.

The proposed semantic is:

FI_MATCH_COMPLETE*
: Indicates that a completion should be generated only after the operation
  has been placed into the application specified buffer.  Operations using
  this completion semantic are dependent on the application at the target
  claiming the message or results.  As a result, match complete may involve
  additional provider level acknowledgements or lengthy delays.  However, this
  completion model enables peer applications to synchronize their execution.

The associated PR is:

https://github.com/ofiwg/libfabric/pull/4303

Note that the PR consolidates and expands the completion levels supported by libfabric into the fi_cq man page.

- Sean