[ofa-general] [PATCH] ib_core: Use weak ordering for data registered memory

[Talpey, Thomas]

At 08:18 AM 10/28/2008, Arnd Bergmann wrote:
>On Monday 27 October 2008, Talpey, Thomas wrote:
>> They can't, right? RDMA operations aren't ordered at all per spec, though
>> there are some architectures/implementations that do.
>> 
>> In fact, one might argue that weak ordering should be the _default_ setting
>> here. It would certainly prevent surprise later.
>
>Well, the problem is that we have existing code out there that assumes
>strict ordering for RDMA, e.g. the eager RDMA option in openmpi. Simply
>changing the Linux implementation breaks that code, which is something
>we don't do if we can avoid.

So, how does openmpi handle this on devices or architectures that don't
provide the placement ordering guarantee? An "eager RDMA option" sounds
suggestive, and unless chosen, wouldn't break anything.

>
>For the mthca device driver, we already have an interface
>(struct mthca_reg_mr) that allows selecting either strict or relaxed
>(aka strong or weak) ordering for a memory region, the default being
>relaxed ordering.
>
>In the long run, I'd like to see something like that for all device drivers,
>so that a user space library can tell the kernel about its requirements.

And vice-versa. If a driver or device cannot provide the requirement, it
needs to communicate that back to the requester.

Tom.