[ofa-general] limit on memory registration

[Dotan Barak]

Rui Machado wrote:
>>> I have a feeling that you refer to the value of max_mr (am i right?)
>>>       
>
>   
>> :) yep sorry.
>> The value for max_mr_size is 18446744073709551615 (can this one be ? )
>> Again, how do I decode this?
>>     
>
>
>   
>>>> Mellanox
>>>> ca type:25218 (vendor_part_id)
>>>> fw_version : 5.1.400 (fw_ver)
>>>> hw_version : a0 (hw_ver)
>>>>
>>>>         
>>> The module parameter "num_mtt" control the size of the above described
>>> table.
>>>       
>> Ok. And is there a limit?
>> And out of curiosity, how does this calculation gets done? I mean, can
>> I take the values and say: Ok, with this num_mtt we can go up to X?
>>     
I guess there is an upper limit (device specific) but since i don't have 
an available data sheet for this device
i can't tell you what it is.
>
> Tried with the module parameter but not sucessfully. I get errors like
>
> ib_mthca 0000:23:00.0: Failed to initialize memory region table, aborting.
> or
> ib_mthca: Invalid value 1048580 for num_mtt in module parameter.
> ib_mthca: Corrected num_mtt to 2097152.
>
> I guess I'm just shooting in the dark :)
>   
I remember that i saw that in the past (i think that the problem was 
kmalloc of more than 128K..)
Can you try to decrease the number of the resources that you don't need 
(MRs/CQs/QPs/SRQs) using the
module parameters, i think that this MAY help you.
> And is there a relation to the max_mr_size above?
>   
No, this limit is the maximum block size that the device can handle.
> Thanks for the patience
> ;)
>   
You are welcome
:)


Dotan