[openib-general] [PATCH 5 of 5] IB/mthca: give reserved MTTs a separate cache line

[Michael S. Tsirkin]

>  > +	/* CPU writes to non-reserved MTTs, while HCA might DMA to reserved mtts */
>  > +	mdev->limits.reserved_mtts = max(dma_get_cache_alignment() / (int)sizeof(u64),
>  > +					 mdev->limits.reserved_mtts);
> 
> I don't follow this -- first of all, what guarantee is there that the
> reserved MTTs end on a cacheline boundary just because they take up
> more than a single cacheline?  It seems this should really be using
> ALIGN() somehow.

Actually, I think that we really must have each of the tables start at
ICM-page aligned addresses. I think this happened to work fine so far
as profile was hard-coded, but with new module option code
this might not be the case anymore.

Since we access some of them from CPU and some from hardware, we really need
them different tables separate dma cache lines too.  ICM-page alignment probably
gives this to us for free, but the following patch makes this assumption explicit.

Pls review.

Warning: untested patch.

Signed-off-by: Michael S. Tsirkin <mst at mellanox.co.il>

---

Index: linux-2.6/drivers/infiniband/hw/mthca/mthca_profile.c
===================================================================
--- linux-2.6.orig/drivers/infiniband/hw/mthca/mthca_profile.c
+++ linux-2.6/drivers/infiniband/hw/mthca/mthca_profile.c
@@ -148,6 +148,10 @@ u64 mthca_make_profile(struct mthca_dev 
 	for (i = 0; i < MTHCA_RES_NUM; ++i) {
 		if (profile[i].size) {
 			profile[i].start = mem_base + total_size;
+			if (mthca_is_memfree(mdev))
+				profile[i].start = ALIGN(profile[i].start,
+							 max(MTHCA_ICM_PAGE_SIZE,
+							     dma_get_cache_alignment()));
 			total_size      += profile[i].size;
 		}
 		if (total_size > mem_avail) {



-- 
MST