[ofa-general] Re: [ewg] [Patch mthca backport] Don't use kmalloc > 128k

[Doug Ledford]

On Jul 23, 2009, at 4:20 AM, Jack Morgenstein wrote:
> On Thursday 16 July 2009 21:08, Doug Ledford wrote:
>> On rhel4 and rhel5 machines, the kmalloc implementation does not
>> automatically forward kmalloc requests > 128kb to __get_free_pages.
>> Please include this patch in all rhel4 and rhel5 backport directories
>> so that we do the right thing in the mthca driver on rhel in regards
>> to kmalloc requests larger than 128k (at least in this code path,
>> there may be others lurking too, I'll forward additional patches if I
>> find they are needed).
>>
>>
> commit a7f18a776785aecb5eb9967aef6f0f603b698ba0
> Author: Doug Ledford <dledford at redhat.com>
> Date:   Thu Jul 16 12:47:55 2009 -0400
>
>    [mthca] Fix attempts to use kmalloc on overly large allocations
>
>    Signed-off-by: Doug Ledford <dledford at redhat.com>

This needs a correct signed-off-by: line.  Mine got added when I put  
it in my local git tree, but the original patch came from Red Hat's  
bugzilla, bug #508902, author David Jeffery <djeffery at redhat.com>

> ----
>
> Roland,
> I think that this patch should be taken into the mainstream kernel,  
> rather
> than just as a backport patch for RHEL.  (We can have a similar  
> patch for mlx4).
> I notice that __get_free_pages(), free_pages(), and get_order() are  
> all in the
> mainstream kernel.
>
> This will fix the 2^20 bits limit on our bitmaps once and for all.
> If you agree, I will post this patch and one for mlx4 on the general  
> list.
>
> Doug posted this patch on the EWG list.
>
> Thanks Doug!
>
> diff --git a/drivers/infiniband/hw/mthca/mthca_mr.c b/drivers/ 
> infiniband/hw/mthca/mthca_mr.c
> index d606edf..312e18d 100644
> --- a/drivers/infiniband/hw/mthca/mthca_mr.c
> +++ b/drivers/infiniband/hw/mthca/mthca_mr.c
> @@ -152,8 +152,11 @@ static int mthca_buddy_init(struct mthca_buddy  
> *buddy, int max_order)
> 		goto err_out;
>
> 	for (i = 0; i <= buddy->max_order; ++i) {
> -		s = BITS_TO_LONGS(1 << (buddy->max_order - i));
> -		buddy->bits[i] = kmalloc(s * sizeof (long), GFP_KERNEL);
> +		s = BITS_TO_LONGS(1 << (buddy->max_order - i)) * sizeof(long);
> +		if(s > PAGE_SIZE)
> +			buddy->bits[i] = (unsigned long *)__get_free_pages(GFP_KERNEL,  
> get_order(s));
> +		else
> +			buddy->bits[i] = kmalloc(s, GFP_KERNEL);
> 		if (!buddy->bits[i])
> 			goto err_out_free;
> 		bitmap_zero(buddy->bits[i],
> @@ -166,9 +169,13 @@ static int mthca_buddy_init(struct mthca_buddy  
> *buddy, int max_order)
> 	return 0;
>
> err_out_free:
> -	for (i = 0; i <= buddy->max_order; ++i)
> -		kfree(buddy->bits[i]);
> -
> +	for (i = 0; i <= buddy->max_order; ++i){
> +		s = BITS_TO_LONGS(1 << (buddy->max_order - i)) * sizeof(long);
> +		if(s > PAGE_SIZE)
> +			free_pages((unsigned long)buddy->bits[i], get_order(s));
> +		else
> +			kfree(buddy->bits[i]);
> +	}
> err_out:
> 	kfree(buddy->bits);
> 	kfree(buddy->num_free);
> @@ -178,10 +185,15 @@ err_out:
>
> static void mthca_buddy_cleanup(struct mthca_buddy *buddy)
> {
> -	int i;
> +	int i, s;
>
> -	for (i = 0; i <= buddy->max_order; ++i)
> -		kfree(buddy->bits[i]);
> +	for (i = 0; i <= buddy->max_order; ++i){
> +		s = BITS_TO_LONGS(1 << (buddy->max_order - i)) * sizeof(long);
> +		if(s > PAGE_SIZE)
> +			free_pages((unsigned long)buddy->bits[i], get_order(s));
> +		else
> +			kfree(buddy->bits[i]);
> +	}
>
> 	kfree(buddy->bits);
> 	kfree(buddy->num_free);


--

Doug Ledford <dledford at redhat.com>

GPG KeyID: CFBFF194
http://people.redhat.com/dledford

InfiniBand Specific RPMS
http://people.redhat.com/dledford/Infiniband




-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 203 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20090723/ca10f5ab/attachment.sig>