[openib-general] Re: netperf for RDS needed
Leonid Arsh
leonida at voltaire.com
Thu Apr 27 07:21:52 PDT 2006
Ranjit
thank you for the patch again. I applied it and succeeded to run.
Looks very nice.
This are the results with for RDS :
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # 10^6bits/sec
262144 8192 10.01 653574 1 4280.59
118784 10.01 653574 4280.59
This are the results without RDS:
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # 10^6bits/sec
262144 8192 10.00 356180 0 2333.90
118784 10.00 211005 1382.63
During the run we get error messages in dmesg on the server side.
Have you seen anything like this?
Please see the dmesg output below:
swapper: page allocation failure. order:1, mode:0x20
Call Trace: <IRQ> <ffffffff801572ae>{__alloc_pages+662}
<ffffffff801184c7>{smp_apic_timer_interrupt+54}
<ffffffff8010e63c>{apic_timer_interrupt+132}
<ffffffff8015a0ff>{cache_grow+288}
<ffffffff8015a4ef>{cache_alloc_refill+419}
<ffffffff80159fb2>{kmem_cache_alloc+87}
<ffffffff8824c01c>{:ib_rds:rds_alloc_buf+16}
<ffffffff8824c0f1>{:ib_rds:rds_alloc_recv_buffer+12}
<ffffffff8824b377>{:ib_rds:rds_post_new_recv+23}
<ffffffff8824bfc3>{:ib_rds:rds_recv_completion+85}
<ffffffff88249b6f>{:ib_rds:rds_cq_callback+87}
<ffffffff8814882b>{:ib_mthca:mthca_eq_int+119}
<ffffffff801102d8>{do_IRQ+50} <ffffffff8010de1e>{ret_from_intr+0}
<ffffffff88148b45>{:ib_mthca:mthca_tavor_interrupt+91}
<ffffffff80151bd5>{handle_IRQ_event+41}
<ffffffff80151ca2>{__do_IRQ+156}
<ffffffff801102d3>{do_IRQ+45} <ffffffff8010de1e>{ret_from_intr+0}
<EOI> <ffffffff8010be87>{mwait_idle+54}
<ffffffff8010be37>{cpu_idle+93}
<ffffffff8052733d>{start_secondary+1131}
Mem-info:
Node 0 DMA per-cpu:
cpu 0 hot: low 0, high 0, batch 1 used:0
cpu 0 cold: low 0, high 0, batch 1 used:0
cpu 1 hot: low 0, high 0, batch 1 used:0
cpu 1 cold: low 0, high 0, batch 1 used:0
cpu 2 hot: low 0, high 0, batch 1 used:0
cpu 2 cold: low 0, high 0, batch 1 used:0
cpu 3 hot: low 0, high 0, batch 1 used:0
cpu 3 cold: low 0, high 0, batch 1 used:0
Node 0 DMA32 per-cpu:
cpu 0 hot: low 0, high 186, batch 31 used:162
cpu 0 cold: low 0, high 62, batch 15 used:39
cpu 1 hot: low 0, high 186, batch 31 used:114
cpu 1 cold: low 0, high 62, batch 15 used:51
cpu 2 hot: low 0, high 186, batch 31 used:36
cpu 2 cold: low 0, high 62, batch 15 used:37
cpu 3 hot: low 0, high 186, batch 31 used:21
cpu 3 cold: low 0, high 62, batch 15 used:31
Node 0 Normal per-cpu: empty
Node 0 HighMem per-cpu: empty
Free pages: 69832kB (0kB HighMem)
Active:40222 inactive:13980 dirty:12 writeback:0 unstable:0 free:17462
slab:176826 mapped:37439 pagetables:1756
Node 0 DMA free:3980kB min:44kB low:52kB high:64kB active:0kB
inactive:0kB present:11224kB pages_scanned:0 all_unreclai
mable? yes
lowmem_reserve[]: 0 990 990 990
Node 0 DMA32 free:65868kB min:4000kB low:5000kB high:6000kB
active:160888kB inactive:55920kB present:1013924kB pages_sc
anned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable?
no
lowmem_reserve[]: 0 0 0 0
Node 0 HighMem free:0kB min:128kB low:128kB high:128kB active:0kB
inactive:0kB present:0kB pages_scanned:0 all_unreclai
mable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 73*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB
1*1024kB 1*2048kB 0*4096kB = 3980kB
Node 0 DMA32: 16225*4kB 17*8kB 8*16kB 2*32kB 1*64kB 1*128kB 0*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 65932kB
Node 0 Normal: empty
Node 0 HighMem: empty
Swap cache: add 72, delete 72, find 33/38, race 0+0
Free swap = 2096460kB
Total swap = 2096472kB
Free swap: 2096460kB
262112 pages of RAM
6465 reserved pages
130652 pages shared
0 pages swap cached
rds: kmem_cache <0xffff810024f52900> returned NULL
netserver: page allocation failure. order:1, mode:0x20
Call Trace: <IRQ> <ffffffff801572ae>{__alloc_pages+662}
<ffffffff8015a0ff>{cache_grow+288}
<ffffffff8015a4ef>{cache_alloc_refill+419}
<ffffffff80159fb2>{kmem_cache_alloc+87}
<ffffffff8824c01c>{:ib_rds:rds_alloc_buf+16}
<ffffffff8824c0f1>{:ib_rds:rds_alloc_recv_buffer+12}
<ffffffff8824b377>{:ib_rds:rds_post_new_recv+23}
<ffffffff8824bfc3>{:ib_rds:rds_recv_completion+85}
<ffffffff88249b6f>{:ib_rds:rds_cq_callback+87}
<ffffffff8814882b>{:ib_mthca:mthca_eq_int+119}
<ffffffff801102d8>{do_IRQ+50} <ffffffff8010de1e>{ret_from_intr+0}
<ffffffff88148b45>{:ib_mthca:mthca_tavor_interrupt+91}
<ffffffff80151bd5>{handle_IRQ_event+41}
<ffffffff80151ca2>{__do_IRQ+156}
<ffffffff801102d3>{do_IRQ+45} <ffffffff8010de1e>{ret_from_intr+0}
<EOI> <ffffffff801f36eb>{copy_user_generic+59}
<ffffffff8824b971>{:ib_rds:rds_recvmsg+566}
<ffffffff802bf257>{sock_common_recvmsg+45}
<ffffffff802bbaec>{sock_recvmsg+271}
<ffffffff8015707d>{__alloc_pages+101}
<ffffffff8032260b>{_read_unlock_irq+6}
<ffffffff801530af>{find_get_page+65}
<ffffffff80144e52>{autoremove_wake_function+0}
<ffffffff802bcdd0>{sys_recvfrom+182}
<ffffffff803225ea>{_spin_unlock_irq+10}
<ffffffff803225e7>{_spin_unlock_irq+7}
<ffffffff803212c9>{thread_return+167}
<ffffffff801350f6>{do_setitimer+333}
<ffffffff8010d85a>{system_call+126}
Mem-info:
Node 0 DMA per-cpu:
cpu 0 hot: low 0, high 0, batch 1 used:0
cpu 0 cold: low 0, high 0, batch 1 used:0
cpu 1 hot: low 0, high 0, batch 1 used:0
cpu 1 cold: low 0, high 0, batch 1 used:0
cpu 2 hot: low 0, high 0, batch 1 used:0
cpu 2 cold: low 0, high 0, batch 1 used:0
cpu 3 hot: low 0, high 0, batch 1 used:0
cpu 3 cold: low 0, high 0, batch 1 used:0
Node 0 DMA32 per-cpu:
cpu 0 hot: low 0, high 186, batch 31 used:160
cpu 0 cold: low 0, high 62, batch 15 used:39
cpu 1 hot: low 0, high 186, batch 31 used:179
cpu 1 cold: low 0, high 62, batch 15 used:55
cpu 1 cold: low 0, high 62, batch 15 used:55
cpu 2 hot: low 0, high 186, batch 31 used:46
cpu 2 cold: low 0, high 62, batch 15 used:37
cpu 3 hot: low 0, high 186, batch 31 used:44
cpu 3 cold: low 0, high 62, batch 15 used:31
Node 0 Normal per-cpu: empty
Node 0 HighMem per-cpu: empty
Free pages: 69484kB (0kB HighMem)
Active:40244 inactive:13258 dirty:1 writeback:1 unstable:0 free:17371
slab:177517 mapped:37438 pagetables:1756
Node 0 DMA free:3980kB min:44kB low:52kB high:64kB active:0kB
inactive:0kB present:11224kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 990 990 990
Node 0 DMA32 free:65504kB min:4000kB low:5000kB high:6000kB
active:160976kB inactive:53032kB present:1013924kB pages_scanned:0
all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB
present:0kB pages_scanned:0 all_unreclaimable?no
lowmem_reserve[]: 0 0 0 0
Node 0 HighMem free:0kB min:128kB low:128kB high:128kB active:0kB
inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 73*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB
1*1024kB 1*2048kB 0*4096kB = 3980kB
Node 0 DMA32: 16188*4kB 0*8kB 1*16kB 1*32kB 1*64kB 1*128kB 0*256kB
1*512kB 0*1024kB 0*2048kB 0*4096kB = 65504kB
Node 0 Normal: empty
Node 0 HighMem: empty
Swap cache: add 72, delete 72, find 33/38, race 0+0
Free swap = 2096460kB
Total swap = 2096472kB
Free swap: 2096460kB
262112 pages of RAM
6465 reserved pages
130733 pages shared
0 pages swap cached
rds: kmem_cache <0xffff810024f52900> returned NULL
More information about the general
mailing list