[ofa-general] uDAPL DTO completion question.

[Jie Cai]

Yes, the variable "target" has been declared volatile. However,
it is a pointer points to "char *rbuf" with type cast, where rbuf been 
allocated
memory with malloc. Will this bring the trouble?

I tried gcc with no optimization, -O2 and -O3 as well, but the program 
still goes
infinitely.

Still haven't figured out where is the problem. Do u have some other 
comments?

Regards,

-- 
Jie Cai



Caitlin Bestler wrote:
> On Fri, May 1, 2009 at 4:24 AM, arkady kanevsky
> <arkady.kanevsky at gmail.com> wrote:
>   
>> Jie,
>> it sounds to me that either the variable is not volatile or compiler
>> optimization
>> causes some problem. I would check for these first.
>> Arkady
>>
>>     
>
> Agreed, it is definitely a caching issue.
>
> Atomics are InfiniBand specific, and there are some fairly complex
> rules that govern
> how much the HCA can do caching. The gotcha is that they basically provide some
> cache coherency guarantees within the context of a connection, but not
> much between
> connections or versus local applications.
>
> That said, it would be rare for HCA caching to be the cause of
> anything worse than
> some unexpected ordering. Adapters cache when they have to, but would
> really rather
> not allocate or track a lot of resources. Updating to real physical
> memory ASAP is much
> simpler.
>
> Compilers, on the other hand, *love* optimizing. The key thing to
> understand is that the
> HCA is another processor, one that is at least as distant as any other
> CPU core. Any
> and all techniques used when sharing memory with another processor apply.
>
> Completions hide all that from the application, just promising that
> specific things are
> coherent when the user invokes the verbs to reap a completion. So
> whenever you do
> without completions you are dealing with an arbitrary multi-processor
> memory coherence
> problem.
>