[openib-general] failure to create an FMR mapping 1K pages on memfree

[Bernadat, Philippe]

Roland is right, I checked were mthca_fmr_alloc() was failing.
Mtts is one page of pointers, so max is 512.
Does work with 512.
I checked, mthca_alloc_fmr returns EINVAL, then ib_create_fmr_pool
returns ENOMEM.

So this isn't a hardware limitation since the Voltaire Stack managed to
handle 1024 pages on the same board.
Is there a way to fix OFED ? 

Philippe

> -----Original Message-----
> From: Roland Dreier [mailto:rdreier at cisco.com] 
> Sent: Tuesday, February 27, 2007 5:33 AM
> To: Or Gerlitz
> Cc: Bernadat, Philippe; openib
> Subject: Re: failure to create an FMR mapping 1K pages on memfree
> 
>  > I have got a report on failure to create FMR mapping 1K 
> pages (that is
>  > 4MB) on memfree.
>  >
>  > I don't have the exact details (ie if Arbel/Sinai / what FW  / etc)
>  > nor which exact check fails in
>  > mthca_fmr_alloc, but what's clear is that the latter 
> function returns
>  > -ENOMEM when attr.max_pages is 1024 and it works fine when
>  > attr.max_pages is 256.
>  >
>  > Is this failure clear to you? if yes, does a HW or FW 
> limit is being
>  > hit or its a driver design issue?
> 
> Is it really returning -ENOMEM?  It seems much more likely that you
> are hitting the code
> 
> 	/* For Arbel, all MTTs must fit in the same page. */
> 	if (mthca_is_memfree(dev) &&
> 	    mr->attr.max_pages * sizeof *mr->mem.arbel.mtts > PAGE_SIZE)
> 		return -EINVAL;
> 
> I guess you could call this limit a driver design issue.
> 
>  - R.
>