[openib-general] page allocation failure with kdapl/mthca

[Hal Rosenstock]

Hi Roland,

With the latest mthca changes (this test worked prior to these changes),
I get the following running kdapl in "loopback":

Jun 21 08:56:09 localhost kernel: DT_Mdep_Thread_: page allocation failure. order:0, mode:0x20
Jun 21 08:56:09 localhost kernel:  [<c01470f2>] __alloc_pages+0x2b2/0x440
Jun 21 08:56:09 localhost kernel:  [<c014d77e>] cache_alloc_debugcheck_after+0x6e/0x1a0
Jun 21 08:56:09 localhost kernel:  [<c014729f>] __get_free_pages+0x1f/0x40
Jun 21 08:56:09 localhost kernel:  [<c010a8ae>] dma_alloc_coherent+0xce/0x100
Jun 21 08:56:09 localhost kernel:  [<c014ddfd>] __kmalloc+0x9d/0xe0
Jun 21 08:56:10 localhost kernel:  [<d08dce3e>] mthca_alloc_wqe_buf+0x44e/0x490 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<c0146bb2>] buffered_rmqueue+0xd2/0x2b0
Jun 21 08:56:10 localhost kernel:  [<d08dd291>] mthca_alloc_qp_common+0x71/0x1b0 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<d08dd634>] mthca_alloc_qp+0xb4/0x270 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<d08e2c16>] mthca_create_qp+0x186/0x2f0 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<d08bebdc>] ib_create_qp+0x2c/0x90 [ib_core]
Jun 21 08:56:10 localhost kernel:  [<d09c2156>] dapl_ib_qp_alloc+0x156/0x377 [ib_dat_provider]
Jun 21 08:56:10 localhost kernel:  [<c014ddfd>] __kmalloc+0x9d/0xe0
Jun 21 08:56:10 localhost kernel:  [<d09cf531>] dapl_evd_qp_async_error_callback+0x0/0xf5 [ib_dat_provider]
Jun 21 08:56:10 localhost kernel:  [<d09ca22e>] dapl_ep_create+0x37b/0x42a [ib_dat_provider]
Jun 21 08:56:10 localhost kernel:  [<d0a25e7c>] DT_Transaction_Main+0x90c/0x21a0 [kdapltest]
Jun 21 08:56:10 localhost kernel:  [<c011180d>] __change_page_attr+0x2d/0x170
Jun 21 08:56:10 localhost kernel:  [<c0111ce8>] kernel_map_pages+0x28/0x70
Jun 21 08:56:10 localhost kernel:  [<c014d196>] cache_free_debugcheck+0x196/0x2d0
Jun 21 08:56:10 localhost kernel:  [<d0a3395f>] DT_Mdep_Thread_Start_Routine+0x1f/0x30 [kdapltest]
Jun 21 08:56:10 localhost kernel:  [<d0a33940>] DT_Mdep_Thread_Start_Routine+0x0/0x30 [kdapltest]
Jun 21 08:56:10 localhost kernel:  [<c0100875>] kernel_thread_helper+0x5/0x10
Jun 21 08:56:10 localhost kernel: ------------[ cut here ]------------
Jun 21 08:56:10 localhost kernel: kernel BUG at mm/page_alloc.c:915!
Jun 21 08:56:10 localhost kernel: invalid operand: 0000 [#1]
Jun 21 08:56:10 localhost kernel: DEBUG_PAGEALLOC
Jun 21 08:56:10 localhost kernel: Modules linked in: kdapltest ib_dat_provider ib_cm ib_at dat ib_ipoib ib_sa ide_cd cdrom lp ipv6 autofs parport_pc parport uhci_hcd ehci_hcd ib_mthca ib_mad ib_core ohci_hcd eepro100 mii evdev usbcore
Jun 21 08:56:10 localhost kernel: CPU:    0
Jun 21 08:56:10 localhost kernel: EIP:    0060:[<c01473c4>]    Not tainted VLI
Jun 21 08:56:10 localhost kernel: EFLAGS: 00010206   (2.6.11.6) 
Jun 21 08:56:10 localhost kernel: EIP is at free_pages+0x24/0x40
Jun 21 08:56:10 localhost kernel: eax: 0009a5a5   ebx: 00000000   ecx: c0f4adf8   edx: 00000000
Jun 21 08:56:10 localhost kernel: esi: cdb1f7f8   edi: 00000000   ebp: 0000001d   esp: c6f23c68
Jun 21 08:56:10 localhost kernel: ds: 007b   es: 007b   ss: 0068
Jun 21 08:56:10 localhost kernel: Process DT_Mdep_Thread_ (pid: 7983, threadinfo=c6f22000 task=c607da90)
Jun 21 08:56:10 localhost kernel: Stack: d08dcd3d cf907c3c 00001000 5a5a5a5a 00000000 c0146bb2 00000000 c101e940 
Jun 21 08:56:10 localhost kernel:        00000000 00000001 00000282 c101e940 c04bddf8 fffffff4 c04bddf8 0000000c 
Jun 21 08:56:10 localhost kernel:        00000040 00b01000 c6f23dd4 c0f4adf8 cdb1f7f8 cdca37f0 d08dd291 cdb1f7f8 
Jun 21 08:56:10 localhost kernel: Call Trace:
Jun 21 08:56:10 localhost kernel:  [<d08dcd3d>] mthca_alloc_wqe_buf+0x34d/0x490 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<c0146bb2>] buffered_rmqueue+0xd2/0x2b0
Jun 21 08:56:10 localhost kernel:  [<d08dd291>] mthca_alloc_qp_common+0x71/0x1b0 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<d08dd634>] mthca_alloc_qp+0xb4/0x270 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<d08e2c16>] mthca_create_qp+0x186/0x2f0 [ib_mthca]
Jun 21 08:56:10 localhost kernel:  [<d08bebdc>] ib_create_qp+0x2c/0x90 [ib_core]
Jun 21 08:56:10 localhost kernel:  [<d09c2156>] dapl_ib_qp_alloc+0x156/0x377 [ib_dat_provider]
Jun 21 08:56:10 localhost kernel:  [<c014ddfd>] __kmalloc+0x9d/0xe0
Jun 21 08:56:10 localhost kernel:  [<d09cf531>] dapl_evd_qp_async_error_callback+0x0/0xf5 [ib_dat_provider]
Jun 21 08:56:10 localhost kernel:  [<d09ca22e>] dapl_ep_create+0x37b/0x42a [ib_dat_provider]
Jun 21 08:56:10 localhost kernel:  [<d0a25e7c>] DT_Transaction_Main+0x90c/0x21a0 [kdapltest]
Jun 21 08:56:10 localhost kernel:  [<c011180d>] __change_page_attr+0x2d/0x170
Jun 21 08:56:10 localhost kernel:  [<c0111ce8>] kernel_map_pages+0x28/0x70
Jun 21 08:56:10 localhost kernel:  [<c014d196>] cache_free_debugcheck+0x196/0x2d0
Jun 21 08:56:10 localhost kernel:  [<d0a3395f>] DT_Mdep_Thread_Start_Routine+0x1f/0x30 [kdapltest]
Jun 21 08:56:10 localhost kernel:  [<d0a33940>] DT_Mdep_Thread_Start_Routine+0x0/0x30 [kdapltest]
Jun 21 08:56:10 localhost kernel:  [<c0100875>] kernel_thread_helper+0x5/0x10
Jun 21 08:56:10 localhost kernel: Code: 8d b4 26 00 00 00 00 85 c0 74 2c 05 00 00 00 40 c1 e8 0c 3b 05 40 aa 3f c0 73 10 8b 0d 50 aa 3f c0 c1 e0 05 01 c8 e9 7c ff ff ff <0f> 0b 93 03 67 6b 31 c0 eb e6 89 f6 c3 eb 0d 90 90 90 90 90 90 

Is this related to the recent changes in mthca ? Does something in kdapl
need to change ? 

Thanks.

-- Hal