[ofiwg] A question on FI_DELIVERY_COMPLETE

[Paul Grun]

What are those mechanisms that MPI and SHMEM use?

Wouldn't it be useful if the requester could simply use REMOTE_CQ_DATA and be assured that the responder wouldn't get the completion until the data had been placed into cache?
-Paul

-----Original Message-----
From: Sur, Sayantan [mailto:sayantan.sur at intel.com] 
Sent: Wednesday, October 21, 2015 9:50 AM
To: Paul Grun; ofiwg at lists.openfabrics.org
Subject: Re: [ofiwg] A question on FI_DELIVERY_COMPLETE

Having the notification at the requester is useful for MPI RMA or SHMEM use cases. This allows MPI/SHMEM to wait for a local event that indicates remote completion. The responder side is passive in these use cases.

Both MPI and SHMEM have different mechanisms to let the responder know when it is able to look at the data.

Thanks,
Sayantan.

From: <ofiwg-bounces at lists.openfabrics.org<mailto:ofiwg-bounces at lists.openfabrics.org>> on behalf of Paul Grun <grun at cray.com<mailto:grun at cray.com>>
Date: Wednesday, October 21, 2015 at 9:33 AM
To: "ofiwg at lists.openfabrics.org<mailto:ofiwg at lists.openfabrics.org>" <ofiwg at lists.openfabrics.org<mailto:ofiwg at lists.openfabrics.org>>
Subject: [ofiwg] A question on FI_DELIVERY_COMPLETE

Here’s my understanding of how FI_DELIVERY_COMPLETE works on the *responder* end:  If you are doing an RMA operation, and the requester uses CQ_REMOTE_DATA to signal the end of the transfer to the responder, and the responder has FI_DELIVERY_COMPLETE set, then the responder won’t get a completion event until the data is actually visible to the responder.

I ask because the man pages imply that FI_DELIVERY_COMPLETE, which is an operation flag, applies only to the requester side.  But it is much less important to notify the requester that data is visible to the responder, than it is to notify the responder itself.

Comments?
-Paul


Cray Inc.
Office:    (503) 620-8757
Mobile:  (503) 703-5382