[openib-general] Re: ibv_dealloc_pd after create + destroy of a AV fails (the resource is busy)

[Roland Dreier]

    Dotan> I'm using svn rev: 2946 on Mellanox HCA 23108.  In user
    Dotan> level: the following scenario fails: allocate a PD create
    Dotan> AV destroy AV deallocate PD

Thanks.  There was a bug in the reference counting for pages used to
hold address vectors.  It's fixed with the change below (already
checked in to svn).

 - R.

--- libmthca/src/ah.c	(revision 2963)
+++ libmthca/src/ah.c	(working copy)
@@ -71,6 +71,8 @@ static struct mthca_ah_page *__add_page(
 		return NULL;
 	}
 
+	page->mr->context = pd->ibv_pd.context;
+
 	page->use_cnt = 0;
 	for (i = 0; i < per_page; ++i)
 		page->free[i] = ~0;
@@ -105,17 +107,18 @@ int mthca_alloc_av(struct mthca_pd *pd, 
 			if (page->use_cnt < ps / sizeof *ah->av)
 				for (i = 0; i < pp; ++i)
 					if (page->free[i])
-						break;
-
-		if (!page)
-			page = __add_page(pd, ps, pp);
+						goto found;
 
+		page = __add_page(pd, ps, pp);
 		if (!page) {
 			free(ah);
 			pthread_mutex_unlock(&pd->ah_mutex);
 			return -1;
 		}
 
+	found:
+		++page->use_cnt;
+
 		for (i = 0, j = -1; i < pp; ++i)
 			if (page->free[i]) {
 				j = ffs(page->free[i]);
@@ -171,6 +174,7 @@ void mthca_free_av(struct mthca_ah *ah)
 		page = ah->page;
 		i = ((void *) ah->av - page->buf) / sizeof *ah->av;
 		page->free[i / (8 * sizeof (int))] |= 1 << (i % (8 * sizeof (int)));
+
 		if (!--page->use_cnt) {
 			if (page->prev)
 				page->prev->next = page->next;