[ofa-general] [PATCH] mlx4: Adjust ownership bit properly in resize_cq when copying over CQEs

[Jack Morgenstein]

mlx4: Adjust ownership bit properly in resize_cq when copying over CQEs

During resize_cq, when copying over unpolled CQEs from the old CQE buffer to
the new buffer, the ownership bit must be set appropriately for the new buffer,
or the ownership bit in the new buffer gets corrupted.

Signed-off-by: Jack Morgenstein <jackm at dev.mellanox.co.il>

---
Roland,
Please queue this one up for 2.6.28.
I don't think there are any kernel apps as yet which are using resize-cq,
but you might consider posting this for 2.6.27 "last stable".

Main user is MPI, which does resize_cq in userspace, and not dynamically
(so it does not hit this bug).
I'll post the equivalent patch for libmlx4.

- Jack
 
Index: infiniband/drivers/infiniband/hw/mlx4/cq.c
===================================================================
--- infiniband.orig/drivers/infiniband/hw/mlx4/cq.c	2008-12-14 17:42:26.000000000 +0200
+++ infiniband/drivers/infiniband/hw/mlx4/cq.c	2008-12-14 17:47:51.000000000 +0200
@@ -325,15 +325,17 @@ static int mlx4_ib_get_outstanding_cqes(
 
 static void mlx4_ib_cq_resize_copy_cqes(struct mlx4_ib_cq *cq)
 {
-	struct mlx4_cqe *cqe;
+	struct mlx4_cqe *cqe, *new_cqe;
 	int i;
 
 	i = cq->mcq.cons_index;
 	cqe = get_cqe(cq, i & cq->ibcq.cqe);
 	while ((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) != MLX4_CQE_OPCODE_RESIZE) {
-		memcpy(get_cqe_from_buf(&cq->resize_buf->buf,
-					(i + 1) & cq->resize_buf->cqe),
-			get_cqe(cq, i & cq->ibcq.cqe), sizeof(struct mlx4_cqe));
+		new_cqe = get_cqe_from_buf(&cq->resize_buf->buf,
+					   (i + 1) & cq->resize_buf->cqe);
+		memcpy(new_cqe, get_cqe(cq, i & cq->ibcq.cqe), sizeof(struct mlx4_cqe));
+		new_cqe->owner_sr_opcode = (cqe->owner_sr_opcode & ~MLX4_CQE_OWNER_MASK) |
+			(((i + 1) & (cq->resize_buf->cqe + 1)) ? MLX4_CQE_OWNER_MASK : 0);
 		cqe = get_cqe(cq, ++i & cq->ibcq.cqe);
 	}
 	++cq->mcq.cons_index;