[ofa-general] RDMA transfers: Buffer status communications?

[Jan Ruffing]

Hello,

when planning a data transfer system using Infiniband's RDMA mechanisms, I stumbled upon the following question: Is there a standard approach to inform the sender after an RDMA_write operation that the receiving buffer has been processed by the receiver and is now ready to receive new data? 

My understanding is as follows:
-  As soon as a IBV_WR_RDMA_WRITE[_WITH_IMM] operation has finished transfering data into the target buffer on the receiver side, a work completion gets put onto the sender side completion queue [and optionally the receiver's completion queue, too].
- The receiver processes the data in the buffer without the sender side noticing
- If the receiver wants to inform the sender that the buffer has been processed and is ready to accept new data, the receiver has to manually send a message to the sender (f.e. by filing a send work request containing some kind of buffer identifier).

Is my understanding of the mechanisms correct? Since locking and unlocking of data receiving buffers is a standard use case in most transport strategies, I wanted to ask if there's a more elegant way to manage this using the Infiniband architecture? Like for example delaying the sender side work completion till the buffer has been processed by the receiver?

Thanks,
Jan


-- 
Jan Ruffing
Software Developer

Motama GmbH
Lortzingstraße 10 · 66111 Saarbrücken · Germany
tel +49 681 940 85 50 · fax +49 681 940 85 49
ruffing at motama.com · www.motama.com

Companies register · district council Saarbrücken · HRB 15249
CEOs · Dr.-Ing. Marco Lohse, Michael Repplinger

This e-mail may contain confidential and/or privileged information. 
If you are not the intended recipient (or have received this e-mail 
in error) please notify the sender immediately and destroy this 
e-mail. Any unauthorized copying, disclosure or distribution of the 
material in this e-mail is strictly forbidden.