[ofa-general] uDAPL DTO completion question.

[arkady kanevsky]

Jie,it sounds to me that either the variable is not volatile or compiler
optimization
causes some problem. I would check for these first.
Arkady

On Fri, May 1, 2009 at 12:35 AM, Jie Cai <Jie.Cai at cs.anu.edu.au> wrote:

> Thanks for the help along my understanding with IB and uDAPL.
>
> Is that possible to spin remote memory of a rdma atomic compare and swap
> (dat_ib_post_cmp_and_swap())?
>
> I have wrote a program that initiator atomic cmp_swap a value to a remote
> memory.
>
> Instead of sending a message to notify the remoter about the completion of
> cmp_swap,
> the remoter actually doing a memory spin to test the update on the memory
> (e.g. while(target == 0);).
>
> However, at remote side, this while loops goes infinitely, and the
> initiator has already received DAT_IB_DTO_EVENT.
>
> I don't really understand what's going on, and what would be a correct way
> to spin memory for checking remote
> write updates.
>
> Any suggestions?
>
> Regards,
> Jie
>
> --
> Jie Cai
>
>
>
>
> Caitlin Bestler wrote:
>
>> On Tue, Mar 31, 2009 at 11:41 PM, Jie Cai <Jie.Cai at cs.anu.edu.au> wrote:
>>
>>
>>> Understood now. A further question is here again.
>>>
>>> To implement software level acknowledgment to inform initiator that data
>>> has been available for remoter, is that possible to use a busy loop at
>>> remote
>>> side to detect the last element of transferring has appear in the memory.
>>>
>>> Or remoter has to wait for the event of recv matching initiator's send,
>>> then
>>> send a message back to initiator as a acknowledgment?
>>>
>>>
>>>
>>
>> There are two issues when spinning on a remote memory update.
>>
>> The first is that packets may be received and processed out of order,
>> especially for iWARP. Therefore the fact that the last byte has been
>> received and placed does not guarantee that the prior packets have
>> been received and placed.
>>
>> More importantly, the order in which updates become visible to a
>> specific software thread can make the order of updates unpredictable
>> to the application.
>>
>> When delivering a completion the Provider is responsible for dealing
>> with both of these problems. So when you reap a completion from the
>> CQ, the operation it represents (and all prior operations) are complete.
>> There are no gaps in received packets, nothing is still sitting on an
>> Adapter buffer waiting to be placed in host memory.
>>
>> If your application does not want to block you can consider polling
>> the cq whether than enabling notifications. But polling memory locations
>> directly should only be done when you're willing to have bus/adapter
>> specific dependencies. You working code might stop working when
>> your network changes, or you install a new Adapter that has a different
>> strategy for optimizing its writes over the PCIe bus.
>>
>>
>


-- 
Cheers,
Arkady Kanevsky
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20090501/f6ae0f6d/attachment.html>