[ofiwg] completion flags as actually defined by OFI

[Hefty, Sean]

> > - The sockets and psm providers do not generate a completion until
> > the remote side has processed the request and acknowledged the data.
> > - Cisco needs to confirm the usnic provider behavior, but it's UD
> > anyway.  I believe it adheres to the description given for
> > completions on unreliable endpoints.
> > -  Verbs does not generate a completion until the data has been
> > acked by the remote side, unless I'm remembering it wrong.
> >
> 
> This is RDMA transport dependent. It might be true for IB but is
> definitively not for iWarp.

iWarp states:

"DDP Message transfer is considered completed when the reliable, in-order transport LLP has indicated that the transfer will occur reliably."

This is why the man pages intentionally try to define the behavior as viewed by the application, and does not define a requirement on the provider or any part of the implementation.