[openib-general][kdapl]: vmalloc instead of kmalloc

[Guy German]

Hi,

James Lentini <mailto:jlentini at netapp.com> wrote:
> On Thu, 4 Aug 2005, Guy German wrote:
> 
>> James,
>> 
>> I see what you mean.
>> The allocation of the event vector is derived from evd->qlen.
>> In DTO ev'd, however, qlen is also the parameter passed to
>> ib_create_cq. Since we don't want to limit DAPL consumers to an
>> unnecessary small completion queue size, maybe we
>> could differentiate between DTO supporting evd's and
>> CONN evd's, when allocating the events vector.
>> 
>> if evd supports CONN only, leave it :
>> 	event = kmalloc(evd->qlen * sizeof *event)
>> (Relying on the consumer he knows what he is doing)
>> if evd is DTO only :
>> 	don't allocate an event buffer, at all
>> if evd supports both :
>> 	event = kmalloc(DEFAULT_4_CONN * sizeof *event)
> 
> And dynamically add additional events up to qlen as needed?
>> 
>> if DEFAULT_4_CONN=256, that's a 3 pages allocation.
>> 
>> How does that sound to you ?
> 
> I'd prefer that the EVDs were uniform. I would worry about bugs
> otherwise. 

But if it's the right thing to do (allocating less, or not allocating at all),
wouldn't it be best to try and fix those bugs ?

> 
> The eventual solution has to support qlen generated events (connection
> request, connection, disconnect, software) if those event types are
> supported by the EVD (even if the EVD is being used for both generated
> events and DTOs).

I think that there might be a mix up in the qlen meaning, for the consumer.
ia_quey returns cq_len parameter, even if the evd in mind is used 
for connection only. In that case the result of qlen (128k), from the hca 
has nothing to do with the real resources availability.

You mentioned before that "allocating an event pool equal to 
the queue length seems like overkill", and I agree.

If you want to support pending events list of more than ~2700, just for CONN events, 
and not use vmalloc, we need some way of doing a few kmallocs
and managing that. Is it really necessary ? wouldn't 256 *pending*
events for CONN. purposes be enough ?

Thanks,
Guy.