[ofa-general] proper use of dev_attr->page_size_cap

[Andy Grover]

Hey all,

So RDS (and srp similarly) are doing this:

rds_ibdev->fmr_page_shift = max(9, ffs(dev_attr->page_size_cap) - 1);

On my mlx4's, page_size_cap is fffffe00, which will always result in
using a 512 byte fmr page size, not really so great.

So ideally we want fmr page shift to match arch's PAGE_SHIFT, but we
have to handle if it must be smaller?

And do this:

rds_ibdev->fmr_page_shift = min(PAGE_SHIFT, fls(PAGE_MASK &
dev_attr->page_size_cap) - 1);

It would also be nice if we could instead do this:

rds_ibdev->fmr_page_shift = PAGE_SHIFT;

? :-)

Regards -- Andy