[openib-general] [mthca] Creation of a SRQ with many WR (> 16K) in kernel level fails

[Or Gerlitz]

Dotan Barak wrote:
> When one tries to create a SRQ with many WR (> 16K WR), creation of the SRQ
> fails.

> static int mthca_alloc_srq_buf(struct mthca_dev *dev, struct mthca_pd *pd,
>                                struct mthca_srq *srq)
>         srq->wrid = kmalloc(srq->max * sizeof (u64), GFP_KERNEL);
>         if (!srq->wrid)
>                 return -ENOMEM;
> which means that creating a SRQ with 16K WRs (or more), the driver will try to
> allocate 16K*8=128K bytes using kmalloc. This is a very high amount of memory
> to be allocated using kmalloc.

mthca_alloc_wqe_buf has the same problem, as it does qp->wrid = 
kmalloc((qp->rq.max + qp->sq.max) * sizeof (u64), GFP_KERNEL);

anyway, the solution that comes into my mind is to disable creating a 
QP/SRQ for which > 128KB allocations are needed. So mthca_query_device() 
will set the max_qp_wr and max_srq_wr attributes to values whose derived 
size still allows to use kmalloc.

Or.