[ofa-general] Re: mlx4: problem with resource limits > 2^20

[Roland Dreier]

 > We're encountering a problem with resource profiles which have elements which exceed 1 Meg
 > (e.g., log_num_qp=21, or log_num_mtt=21 as module options for mlx4_core)
 > 
 > Many kernels allow kmalloc of only up to 128KB (which can support a 1 MB bitmap).  If the
 > resource max is greater than 1 Meg, the kmalloc will fail.

Yes, and mthca has the same issue too...

 > 1. Change the bitmap allocator and buddy systems to use a 2-level scheme.
 > 2. Use vmalloc for allocations greater than 128K, and note that for x86 systems you cannot specify more
 >    than 1M for any resource in the profile.
 > 3. Do nothing, and just note that cannot allocate more than 1M of any resource for ANY system.
 > 
 > My own preference is 2 (with maybe some test to determine just what the crossover point is, rather than
 > just having 128K as a defined constant) -- or, given some time, 1 (which is a more general and scalable solution).

I think doing "if (something) kmalloc() else vmalloc()" is really
ugly, so I would really prefer to see #1.

 - R.