[ofa-general] Re: Write combining on PPC64

[Shirley Ma]

Sorry Jack, I was on vacation. You already got the answer, I think. Yes,
you can use pgprot_noncached_wc().

BTW, can you please modify the connectX set_4K_mtu parameter to 644
instead of 444 in OFED-1.5? I found for the large message size, if we
don't set up 4K MTU size (SM and switch should support 4K MTU too), the
performance could drop 5%.

Thanks
Shirley

On Sun, 2009-06-28 at 12:21 +0300, Jack Morgenstein wrote: 
> Hi Shirley,
> 
> I was reviewing write-combining for the PPC on kernel 2.6.30, and noticed the following
> in file arch/powerpc/include/asm/pgtable.h:
> 
> #define _PAGE_CACHE_CTL (_PAGE_COHERENT | _PAGE_GUARDED | _PAGE_NO_CACHE | \
>                          _PAGE_WRITETHRU)
> ...
> #define pgprot_noncached_wc(prot) (__pgprot((pgprot_val(prot) & ~_PAGE_CACHE_CTL) | \
>                                             _PAGE_NO_CACHE))
> 
> It seems to me that this gives the same result as the fix I put in OFED 1.5 for the PPC:
> (kernel_patches/fixes/mlx4_0010_add_wc.patch):
> 
> +pgprot_t pgprot_wc(pgprot_t _prot)
> +{
> +       return __pgprot((pgprot_val(_prot) | _PAGE_NO_CACHE) &
> +                                    ~(pgprot_t)_PAGE_GUARDED);
> +}
> 
> ( That is, PAGE_NO_CACHE set, and _PAGE_GUARDED cleared).
> 
> Differences are that _PAGE_COHERENT and _PAGE_WRITETHRU are also cleared in pgprot_noncached_wc()
> and the patch I put in is only for PPC64, not all PPCs.
> 
> Questions:
> 1. Can I use the pgprot_noncached_wc() macro to implement write-combining on all PPCs? (it is defined
>    thus in arch/powerpc/include/asm/pgtable.h for all PPC platforms)
> 
> 2. Is there any reason that there is no define:
> 
>     #define  pgprot_writecombine  pgprot_noncached_wc
>    
>    in file arch/powerpc/include/asm/pgtable.h  (so that the general
>      pgprot_writecombine() call can be used for PPCs as well).
> 
> -Jack