[ofa-general] Re: movnt

[Michael S. Tsirkin]

> Quoting Roland Dreier <rdreier at cisco.com>:
> Subject: Re: movnt
> 
>  > So we can map the device memory with WB or WT semantics, and movnt will enable
>  > WC. And the nice thing about this trick, is that both WB and WT *are already
>  > programmed into PAT after reset*, which means that we can use them for pages we
>  > map for userspace, without stepping on anyone's toes or waiting for
>  > the generic in-kernel support for WC to materialize.
> 
> I'm not sure whether this is much of an advantage.  There's no generic
> way to map memory with WB that I know of.

I think we just need to avoid setting noncacheable flag -
this does not sound too bad.

>  I don't think that setting
> a PAT entry for WC is the hold-up

I think it is - editing PAT will affect all of the system, you start
hittig various errata ...

> -- the problem is more in the right
> infrastructure for pgprot_xxx().  I don't think it's very nice to have
> #ifdef __x86_64__ in a driver.

Not nice, but I think doing it in a platform-dependent way first
will be a good way for whoever implements the portable infrastructure
to see the patterns. So far, most people think about X as the main
user for the infrastructure-to-emerge - an easier to experiment with
user such as libmlx4 will be a good testcase I think.

Look at drivers/infiniband/hw/ipath/ipath_file_ops.c
for example - so why make exception for mthca?

Isn't the following a good start?

        } else if (vma->vm_pgoff == 1 && dev->dev->caps.bf_reg_size != 0) {
-                /* FIXME want pgprot_writecombine() for BlueFlame pages */
+#if defined(__x86_64__)
                vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+#endif
                if (io_remap_pfn_range(vma, vma->vm_start,
                                       to_mucontext(context)->uar.pfn +
                                       dev->dev->caps.num_uars,
                                       PAGE_SIZE, vma->vm_page_prot))
                        return -EAGAIN;
        } else

This does not look too bad, does it?
All we need to do is requite libmlx4 to always use movnti on x86_64.

-- 
MST