[openib-general] ib_create_cq memory leak?

Viswanath Krishnamurthy viswa.krish at gmail.com
Wed Sep 21 13:14:36 PDT 2005 

I ran into this issue when using the kernel API to create CQ's. In order to
reproduce this problem, I wrote
a small kernel module which creates 4K CQ's and destroys them. After running
the test (8-10 times), I saw
create_cq error with error -12 (ENOMEM).

I am attaching the test module source code with Makefiles

[root src]# svn info (Latest code)
Path: .
URL: https://openib.org/svn/gen2/trunk/src
Repository UUID: 21a7a0b7-18d7-0310-8e21-e8b31bdbf5cd
Revision: 3512
Node Kind: directory
Schedule: normal
Last Changed Author: halr
Last Changed Rev: 3511
Last Changed Date: 2005-09-21 08:57:38 -0700 (Wed, 21 Sep 2005)


To compile the code, change the KERNELSRC variable in mysock.mak to point to
your kernel source tree

#make -f mysock.mak

#insmod mysock.ko

To run the test

#echo 1 > /dev/mysock

After 8-10 times of running the above, you will see a -12 error on the
console.

This problem does not occur when you create a single CQ and destroy it
immediately in a loop (I tried 100000 times).
This occurs when you create 4K CQ's and then destroy it.


ib_mthca 0000:05:00.0: Mapped page at 362f9000 to 7e000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 362fa000 to 41000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35d10000 to 7d000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35d11000 to 42000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35f27000 to 7c000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35f28000 to 43000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3593f000 to 7b000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35940000 to 44000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35b56000 to 7a000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35b57000 to 45000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3556d000 to 79000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3556e000 to 46000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35785000 to 78000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35786000 to 47000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 26040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 20040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 25840000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3519b000 to 77000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3519c000 to 48000 for ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 7e000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 7d000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 7c000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 7b000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 7a000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 79000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 78000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 25840000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 20040000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 48000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 77000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 26040000 from ICM.
ib_mthca 0000:05:00.0: Mapped page at 35b03000 to 76000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 362ba000 to 75000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35d2c000 to 74000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35c83000 to 73000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35b99000 to 72000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35db0000 to 71000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 356c5000 to 70000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 26040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 20040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 25840000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35adc000 to 6f000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35add000 to 49000 for ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 76000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 75000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 74000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 73000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 72000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 71000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 70000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 25840000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 20040000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 49000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 6f000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 26040000 from ICM.
ib_mthca 0000:05:00.0: Mapped page at 362cf000 to 6e000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35a0f000 to 6d000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35070000 to 6c000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35e83000 to 6b000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35bd8000 to 6a000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 351ef000 to 69000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35c44000 to 68000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 26040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 20040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 25840000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 355db000 to 67000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 355dc000 to 4a000 for ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 6e000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 6d000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 6c000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 6b000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 6a000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 69000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 68000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 25840000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 20040000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 4a000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 67000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 26040000 from ICM.
ib_mthca 0000:05:00.0: Mapped page at 35d15000 to 66000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 359d0000 to 65000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 36121000 to 64000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35544000 to 63000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35c15000 to 62000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35e2c000 to 61000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 36342000 to 60000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 26040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 20040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 25840000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 358d7000 to 5f000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 358d8000 to 4b000 for ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 66000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 65000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 64000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 63000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 62000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 61000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 60000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 25840000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 20040000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 4b000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 5f000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 26040000 from ICM.
ib_mthca 0000:05:00.0: Mapped page at 35b83000 to 5e000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35762000 to 5d000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3595b000 to 5c000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3619e000 to 5b000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3574f000 to 5a000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 354a3000 to 59000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35db8000 to 58000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 26040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 20040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 25840000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3520e000 to 57000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3520f000 to 4c000 for ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 5e000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 5d000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 5c000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 5b000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 5a000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 59000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 58000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 25840000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 20040000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 4c000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 57000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 26040000 from ICM.
ib_mthca 0000:05:00.0: Mapped page at 35b80000 to 56000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35f01000 to 55000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 361ba000 to 54000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3518d000 to 53000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3560e000 to 52000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 362c1000 to 51000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 35fb2000 to 50000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 26040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 20040000 for ICM.
ib_mthca 0000:05:00.0: Mapped 1 chunks/256 KB at 25840000 for ICM.
ib_mthca 0000:05:00.0: Mapped page at 3500b000 to 4f000 for ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 4f000 from ICM.
mysock: CQ creation err -12, iter 4091
ib_mthca 0000:05:00.0: Unmapping 1 pages at 56000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 55000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 54000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 53000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 52000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 51000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 25840000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 20040000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 1 pages at 50000 from ICM.
ib_mthca 0000:05:00.0: Unmapping 64 pages at 26040000 from ICM.
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507
mysock: CQ creation err -12, iter 507


-Viswa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050921/813fbf44/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mysock.c
Type: application/octet-stream
Size: 3373 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050921/813fbf44/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mysock.mak
Type: application/octet-stream
Size: 92 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050921/813fbf44/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Makefile
Type: application/octet-stream
Size: 64 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050921/813fbf44/attachment-0002.obj>

More information about the general mailing list