[ofa-general] Re: why is CPU util/service demand so much higher withSDP than TCP?

[Scott Weitzenkamp (sweitzen)]

# modinfo ib_sdp
filename:
/lib/modules/2.6.16.21-0.8-smp/updates/kernel/drivers/infiniband
/ulp/sdp/ib_sdp.ko
author:         Michael S. Tsirkin
description:    InfiniBand SDP module
license:        Dual BSD/GPL
vermagic:       2.6.16.21-0.8-smp SMP gcc-4.1
depends:        ib_core,rdma_cm
srcversion:     91793E4825DEBC7A2DA9366
parm:           top_mem_usage:Top system wide sdp memory usage for recv
(in MB).
 (int)
parm:           rcvbuf_scale:Receive buffer size scale factor. (int)
parm:           send_poll_thresh:Send message size thresh hold over
which to sta
rt polling. (int)
parm:           recv_poll:How many times to poll recv. (int)
parm:           send_poll:How many times to poll send. (int)
parm:           recv_poll_miss:How many times recv poll missed. (int)
parm:           recv_poll_hit:How many times recv poll helped. (int)
parm:           send_poll_miss:How many times send poll missed. (int)
parm:           send_poll_hit:How many times send poll helped. (int)
parm:           data_debug_level:Enable data path debug tracing if > 0.
(int)
parm:           debug_level:Enable debug tracing if > 0. (int)

Scott 

> -----Original Message-----
> From: general-bounces at lists.openfabrics.org 
> [mailto:general-bounces at lists.openfabrics.org] On Behalf Of Rick Jones
> Sent: Friday, April 27, 2007 12:36 PM
> To: Michael S. Tsirkin
> Cc: general at lists.openfabrics.org
> Subject: [ofa-general] Re: why is CPU util/service demand so 
> much higher withSDP than TCP?
> 
> Michael S. Tsirkin wrote:
> >>Quoting Rick Jones <rick.jones2 at hp.com>:
> >>Subject: why is CPU util/service demand so much higher with 
> SDP than TCP?
> >>
> >>So, while playing around with my new netperf SDP_RR test 
> I've noticed that 
> >>a single-byte _RR test over SDP has a much higher 
> transactions per second 
> >>(ie lower latency) than over TCP over the same HCA, but the 
> CPU utilization 
> >>is _very_ much higher and the service demand (cpu per 
> transaction) as well. 
> >>CPU util being higher makes sense with a higher transaction 
> rate, but not 
> >>the increased service demand - well at least not to my 
> experience thusfar.
> > 
> > 
> > That's expected.
> > SDP by default uses polling aggressively to trade off 
> service demand for latency.
> > You can play with recv_poll module parameter to tune that.
> 
> Ah, so it is doing a sit and spin waiting for traffic.  I'll 
> see about the 
> recv_poll module parm - is it a binary or other?
> 
> rick jones
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit 
> http://openib.org/mailman/listinfo/openib-general
>