[Openib-windows] Adding an environment variable for determining the initial CQ size

[Tzachi Dar]

Fab,

We do plan to add resize CQ support but since it is quite a large  and
risky patch we prefer not to do that before the coming release. 
The WSD provider implementation will open another CQ in case there is no
more place available and will not free the CQ if it is empty we may get
to a leakage scenario. A larger CQ helps to reduce this. There is no
limitation for large cluster other then this possible leakage.

Once again, we will defiantly add resize CQ to MTHCA but I doubt if it
will be ready for the coming release.

Each CQE entry is 32B. This mean that the memory allocated per CQ is 32
* 512 * 32 = 512KB

Thanks
Tzachi 

> -----Original Message-----
> From: ftillier.sst at gmail.com [mailto:ftillier.sst at gmail.com] 
> On Behalf Of Fabian Tillier
> Sent: Thursday, July 13, 2006 1:27 AM
> To: Tzachi Dar
> Cc: openib-windows at openib.org
> Subject: Re: [Openib-windows] Adding an environment variable 
> for determining the initial CQ size
> 
> Hi Tzachi,
> 
> On 7/12/06, Tzachi Dar <tzachid at mellanox.co.il> wrote:
> >
> > Hi Fab,
> >
> > The following patch does two things:
> > 1) It adds a new environment variable (IBWSD_INITIAL_CQ_SIZE) that 
> > controls the number of QPs that will use the same CQ.
> 
> Is this worth doing?  Isn't the MTHCA driver going to have CQ 
> resize support soon?  I was under the impression that the 
> existing code to size the CQ to support 100 QPs was temporary 
> until MTHCA was updated.
> This change makes me think that resize CQ support is not 
> going to happen anytime soon.
> 
> > 2) It sets the default value from 100 to 500. We believe that this 
> > will allow the users to work on bigger Clusters while the 
> increase in 
> > memory is negligible for a modern system.
> 
> Can you quantify the memory footprint of a CQ sized for 500 
> QPs, each of which has 32 entries (so 16000 entries)?  What's 
> the size of each CQE?
> 
> Also, what is preventing the existing implementation from 
> workign on large clusters?  I don't understand what the 
> problem we're trying to solve is.
> 
> As to the patch itself, it would be better to rename 
> initial_cq_size to initial_qp_per_cq or something like that, 
> since that's what's being controlled here.  This applies to 
> both the environment variable and the internal global variable.
> 
> Thanks,
> 
> - Fab
> 
> 
>