[ofa-general] max_mr limit

[Rajouri Jammu]

On Fri, Mar 21, 2008 at 2:08 PM, Roland Dreier <rdreier at cisco.com> wrote:

>  > I burnt a mem-free f/w on the card and this time I'm getting a
> different
>  > error when trying to set num_mtt to 2097152.
>
>  > I did:
>
>  > modprobe ib_mtcha num_mpt=262144 num_mtt=2097152
>
>  >  kernel: ib_mthca 0000:01:00.0: Failed to initialize memory region
> table, aborting.
>
> You may not have enough contiguous memory for the buddy allocator
> bitmaps for the MTT.  What does /proc/buddyinfo show when this
> happens?
>

I have 32GB of physical memory in the system.

cat /proc/buddyinfo
Node 0, zone      DMA      3      5      3      2      4      0      2
 0      2      0      2
Node 0, zone    DMA32      1      0      0      2      5      4      2
 2      3      3    486
Node 0, zone   Normal     80     38      8    189    335     46     40
 3      3      1   7366

No idea how to interpret that.

Can I change some system parameter so that it gets the required memory?


>  > Roland, when your attempt succeeded were you using a mem-free HCA or
> the
>  > 256MB version?
>
> mem-free.  With the fixes I just posted, I am able to load the driver
> with num_mtt=33554432 and have it work in fact.
>
> It's not clear to me why you need such crazy values.  Having 2M MTT
> segments means you can map 64GB of memory with 4KB pages... do you
> really need that much memory pinned for IB operations?
>

I thought 2M MTTs will allow to map up to 8GB of memory?
2^21*4K. Is that not correct?



>
>  - R.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080321/b3dab803/attachment.html>