[ofiwg] completion flags as actually defined by OFI

[Jason Gunthorpe]

On Tue, Apr 14, 2015 at 07:27:55PM +0000, Hefty, Sean wrote:

> > > The point of my post wasn't even about whether these 3 flags were
> > > desirable.  It was whether a 4th flag
> > > (FI_DATA_VISIBLE_AT_REMOTE_SIDE_BUT_NOT_YET_IN_FINAL_DESTINATION_BUFFER)
> > > was needed.
> > 
> > You mean, the WR is guarenteed to complete on the peer, but the peer
> > may not have seen it yet?
> > 
> > I think that is very useful, and should replace FI_TRANSMIT_COMPLETE.
> 
> The completion semantic being proposed did not make any claim about
> whether the operation was guaranteed to complete on the peer.  Based
> on the proposal, it seems to allow for the possibility that the
> operation could actually fail.

I think the weak semantic of 'under normal opreation' is the best that
can be done here. Making that broad enough to include clean shutdown
is the key utility.

I would say that the guarentee should be for all clean shutdowns and
all posibilities of network packet loss/corruption/etc.

It can exclude QP errors, app crashing, node failing, etc.

> I believe the providers support this guarantee.

Including the clean shutdown case?

Jason