[openib-general] Re: libsdp running nearly fine

[Michael S. Tsirkin]

Quoting r. Xavier Grave <grave at ipno.in2p3.fr>:
> Subject: libsdp running nearly fine
> 
> Hi all,
> 
> I have setup libsdp and it works quite well except if I try to send
> buffer with a size > 5100 bytes I get this kind of kernel messages :
> Unable to handle kernel paging request for data at address
> 0xd000080080085cc0   
> Faulting instruction address:
> 0xd0000000001dd3b4                                
> Oops: Kernel access of bad area, sig: 7
> [#1]                                    
> SMP NR_CPUS=32 NUMA PSERIES
> LPAR                                                
> Modules linked in: ipv6 nfsd exportfs nfs_acl lockd sunrpc ib_uverbs
> psmouse idv
> NIP: D0000000001DD3B4 LR: C00000000022C808 CTR:
> D0000000001DD25C                
> REGS: c0000000b27d7330 TRAP: 0300   Not tainted
> (2.6.16-rc2)                   
> MSR: 8000000000009032 <EE,ME,IR,DR>  CR: 24000488  XER:
> 00000010                
> DAR: D000080080085CC0, DSISR:
> 0000000042000000                                  
> TASK = c000000004a12040[2760] 'client' THREAD: c0000000b27d4000 CPU:
> 3          
> GPR00: 0000000000020000 C0000000B27D75B0 D0000000002014A8
> C0000000042A0B00      
> GPR04: C0000000049B3BA0 0000000000000002 0000000000481000
> C0000000004CE588      
> GPR08: 0000000000020033 0000000000020033 D000080080085CC0
> 00000000000000F0      
> GPR12: 0000200000000000 C0000000003BC100 00000000100D0000
> 0000000000000000      
> GPR16: 0000000000000000 0000000010197EA8 0000000000000001
> C0000000B27D7C98      
> GPR20: 0000000000000000 C0000000B27D7B08 C0000000B1AD1E60
> C0000000049B3BA0      
> GPR24: 0000000000000002 C00000000474FC80 C000000007323A20
> 8000000000009032      
> GPR28: C00000000474FC98 C000000007323A00 C000000000407420
> C000000007323A10      
> NIP [D0000000001DD3B4] .mthca_tavor_map_phys_fmr+0x158/0x190
> [ib_mthca]         
> LR [C00000000022C808] .ib_fmr_pool_map_phys
> +0x2a4/0x4a8                         
> Call
> Trace:                                                                     
> [C0000000B27D75B0] [C00000000022C5C4] .ib_fmr_pool_map_phys+0x60/0x4a8
> (unrelia)
> [C0000000B27D7670] [C00000000024E7B0] .sdp_iocb_register
> +0x5c/0x11c             
> [C0000000B27D7700] [C000000000253A8C] .sdp_send_data_queue_test
> +0x624/0xd7c
> [C0000000B27D7820] [C000000000254220] .sdp_send_data_queue
> +0x3c/0xb0            
> [C0000000B27D78C0] [C000000000255078] .sdp_inet_send
> +0x5d8/0xc9c                
> [C0000000B27D7A10] [C00000000025BEEC] .sock_sendmsg
> +0x114/0x15c                 
> [C0000000B27D7C10] [C00000000025CACC] .sys_sendto
> +0xd0/0x110                    
> [C0000000B27D7D90] [C00000000027BE9C] .compat_sys_socketcall
> +0x148/0x214        
> [C0000000B27D7E30] [C0000000000086F8] syscall_exit
> +0x0/0x40                     
> Instruction
> dump:                                                               
> 81230020 e9430048 396000f0 90030040 60000000 60000000 60000000
> e8080636         
> 7d290214 79280020 91030024 91030020 <996a0000> 7c0004ac 2f850000
> 78a90020
> 
> kernel is compiled with infiniband svn drivers, power5 based server.
> I allocate the memory with memalign and getpagesize, I compiled the
> ib_sdp with zero copy buffer on.
> Do I need to remove these options ?
> 
> xavier

Yes, zcopy is currently Intel only - I dont use the DMA API properly.

-- 
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies