[ofa-general] Re: [PATCH] IB/ehca: Make sure user pages are from hugetlb before using MR large pages

[Joachim Fenkes]

Roland Dreier <rdreier at cisco.com> wrote on 13.09.2007 06:33:45:

>  > -#define HCA_CAP_MR_PGSIZE_4K  1
>  > -#define HCA_CAP_MR_PGSIZE_64K 2
>  > -#define HCA_CAP_MR_PGSIZE_1M  4
>  > -#define HCA_CAP_MR_PGSIZE_16M 8
>  > +#define HCA_CAP_MR_PGSIZE_4K  0x80000000
>  > +#define HCA_CAP_MR_PGSIZE_64K 0x40000000
>  > +#define HCA_CAP_MR_PGSIZE_1M  0x20000000
>  > +#define HCA_CAP_MR_PGSIZE_16M 0x10000000
> 
> Not sure I understand what this has to do with things... is this an
> unrelated fix?

Kinda. I can put it into its own patch if you want.
 
> I would suggest extending ib_umem_get() to check the vmas and adding a
> member to struct ib_umem to say whether the memory is entirely covered
> by hugetlb pages or not.

I like that approach - one patch coming right up! =)
 
>  > +      default: /* out of mem */
>  > +         ib_mr = ERR_PTR(-ENOMEM);
>  > +         goto reg_user_mr_exit1;
> 
> It seems like it would be better to just assume the memory is not from
> a hugetlb is ehca_is_mem_hugetlb() fails its memory allocation and
> fall back to the PAGE_SIZE case rather than failing entirely.

If ehca_is_mem_hugetlb() runs out of memory, ehca_reg_mr() is rather 
unlikely to get the memory, but it's worth a try, I'll give you that. I'll 
make the umem patch work that way.

Joachim