[ofa-general] Problem with ib_dma_map_single

[Max Lapan]

Hello,

currently, I'm trying to write small sample kernel module which
performs verbs communication in UD mode. Almost everything is
complete, but when I am trying to map allocated memory for DMA, I get
GPF:

general protection fault: 0000 [1] SMP
last sysfs file: /sys/devices/pci0000:80/0000:80:11.0/0000:86:01.1/irq
CPU 0
Modules linked in: verbs(N+) bridge stp netbk blkbk blktap xenbus_be
rdma_ucm(N) ib_ucm(N) ib_srp(N) scsi_transport_srp scsi_tgt ib_sdp(N)
rdma_cm(N) iw_cm(N) ib_addr(N) ib_ipoib(N) ib_cm(N) ib_sa(N) ipv6
ib_uverbs(N) ib_umad(N) ib_ipath(N) mlx4_ib(N) ib_mad(N) ib_core(N)
fuse ext2 loop sg mptctl e1000 rtc_cmos rtc_core rtc_lib serio_raw
pcspkr shpchp k8temp(N) pci_hotplug mlx4_core(N) forcedeth joydev
8250_pnp cfi_cmdset_0001(N) cfi_util(N) i2c_nforce2 jedec_probe(N)
i2c_core cfi_probe(N) gen_probe(N) button ck804xrom(N) mtd chipreg(N)
map_funcs(N) 8250 serial_core usbhid hid ff_memless linear ohci_hcd
sd_mod crc_t10dif ehci_hcd usbcore dm_snapshot xenblk cdrom xennet edd
dm_mod ext3 mbcache jbd fan ide_pci_generic amd74xx ide_core
ata_generic pata_amd libata dock mptsas mptscsih mptbase
scsi_transport_sas scsi_mod thermal processor thermal_sys hwmon
Supported: No
Pid: 4027, comm: insmod Tainted: G          2.6.27.23-0.1-xen #1
RIP: e030:[<ffffffffa04a248d>]  [<ffffffffa04a248d>]
verbs_add_device+0x497/0xd9e [verbs]
RSP: e02b:ffff880070ea7c28  EFLAGS: 00010286
RAX: ec83485355544155 RBX: ffff880073556000 RCX: 0000000000000002
RDX: 0000000000010000 RSI: ffff8800693f0000 RDI: ffff880073556000
RBP: ffffffffa04a48c0 R08: ffff880074803ae0 R09: 0000000074ac82e0
R10: 00008d8d00000000 R11: 000200d08068c8e8 R12: ffffffffa0200000
R13: 0000000000000000 R14: 00007fff478f978e R15: 0000000000000002
FS:  00007f753f8e56f0(0000) GS:ffffffff80761080(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process insmod (pid: 4027, threadinfo ffff880070ea6000, task ffff8800693a0640)
Stack:  0000000000000000 0000000000000000 0000000000000000 0000000000000000
 0000000000000000 0000000000000000 0000000000000000 0000000000000000
 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Call Trace:
 [<ffffffffa02f1462>] ib_register_client+0x5b/0x88 [ib_core]
 [<ffffffffa020006e>] verbs_init+0x6e/0x9e [verbs]
 [<ffffffff80208042>] _stext+0x42/0x14a
 [<ffffffff80255cad>] sys_init_module+0xa0/0x1ba
 [<ffffffff8020b3b8>] system_call_fastpath+0x16/0x1b
 [<00007f753f45576a>] 0x7f753f45576a


Code: 00 48 8b 80 98 02 00 00 48 8b 40 08 8b 8c 24 3c 01 00 00 48 8b
94 24 40 01 00 00 48 8b b4 24 48 01 00 00 48 8b bc 24 50 01 00 00 <ff>
d0 48 89 44 24 08 e9 8b 01 00 00 8b 94 24 3c 01 00 00 48 8b
RIP  [<ffffffffa04a248d>] verbs_add_device+0x497/0xd9e [verbs]
 RSP <ffff880070ea7c28>
---[ end trace 0975eb2c30b98904 ]---

This error caused by this piece of code:

len = 2048;
buf = kzalloc (len, GFP_KERNEL);
key = ib_dma_map_single (dev, buf, len, DMA_TO_DEVICE);

I've tried to use different GFP_XX flags, but without success. The
similar code in ipoib and srp drivers works well, but, maybe I am
missing something. I also tried to use ib_dma_alloc_coherent, but got
the same error.

Are there any restrictions for ib_dma_map_single call? Must I
initialize something to call it? On this machine I have SLES11 with
OFED 1.4.1 installed. All IB stuff works well (IPoIB, SRP, verbs
benchmarks, etc).

--
wbr, Max Lapan