[ofa-general] max_mr limit

[Roland Dreier]

 > >  >  kernel: ib_mthca 0000:01:00.0: Failed to initialize memory region
 > > table, aborting.
 > >
 > > You may not have enough contiguous memory for the buddy allocator
 > > bitmaps for the MTT.  What does /proc/buddyinfo show when this
 > > happens?
 > >
 > 
 > I have 32GB of physical memory in the system.
 > 
 > cat /proc/buddyinfo
 > Node 0, zone      DMA      3      5      3      2      4      0      2
 >  0      2      0      2
 > Node 0, zone    DMA32      1      0      0      2      5      4      2
 >  2      3      3    486
 > Node 0, zone   Normal     80     38      8    189    335     46     40
 >  3      3      1   7366
 > 
 > No idea how to interpret that.

Actually looks like you have lots of contiguous memory.

I guess you could add printk()s before each return/goto err statement
in mthca_init_mr_table() and see exactly what is failing.

 > I thought 2M MTTs will allow to map up to 8GB of memory?
 > 2^21*4K. Is that not correct?

Not positive I'm remembering correctly, but I'm pretty sure that mthca
accounts for MTTs in segments of 8 entries.  So you get another factor
of 8 in your calculation, and 2M MTTs will cover 64GB.