[ofa-general] [PATCH 5 of 5] mlx4: Do not allocate an extra (unneeded) CQE when creating a CQ

[Jack Morgenstein]

mlx4: Do not allocate an extra (unneeded) CQE when creating a CQ.

The extra CQE can cause a huge waste of memory if requesting
a power-of-2 number of CQEs.

Leave create_cq for userspace CQs as before, to avoid breaking ABI.
(Handle this in separate libmlx4 patch)

Signed-off-by: Jack Morgenstein <jackm at dev.mellanox.co.il>

diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 8bf44da..8a1ccc4 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -108,7 +108,13 @@ struct ib_cq *mlx4_ib_create_cq(struct ib_device *ibdev, int entries, int vector
 	if (!cq)
 		return ERR_PTR(-ENOMEM);
 
-	entries      = roundup_pow_of_two(entries + 1);
+	/* eliminate using extra CQE (for kernel space).
+	 * For userspace, do in libmlx4, so that don't break ABI.
+	 */
+	if (context)
+		entries      = roundup_pow_of_two(entries + 1);
+	else
+		entries      = roundup_pow_of_two(entries);
 	cq->ibcq.cqe = entries - 1;
 	buf_size     = entries * sizeof (struct mlx4_cqe);
 	spin_lock_init(&cq->lock);
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c
index 89b3f0b..d34b61b 100644
--- a/drivers/net/mlx4/main.c
+++ b/drivers/net/mlx4/main.c
@@ -141,12 +141,7 @@ static int mlx4_dev_cap(struct mlx4_dev *dev, struct mlx4_dev_cap *dev_cap)
 	dev->caps.max_sq_desc_sz     = dev_cap->max_sq_desc_sz;
 	dev->caps.max_rq_desc_sz     = dev_cap->max_rq_desc_sz;
 	dev->caps.num_qp_per_mgm     = MLX4_QP_PER_MGM;
-	/*
-	 * Subtract 1 from the limit because we need to allocate a
-	 * spare CQE so the HCA HW can tell the difference between an
-	 * empty CQ and a full CQ.
-	 */
-	dev->caps.max_cqes	     = dev_cap->max_cq_sz - 1;
+	dev->caps.max_cqes	     = dev_cap->max_cq_sz;
 	dev->caps.reserved_cqs	     = dev_cap->reserved_cqs;
 	dev->caps.reserved_eqs	     = dev_cap->reserved_eqs;
 	dev->caps.reserved_mtts	     = DIV_ROUND_UP(dev_cap->reserved_mtts,