[ofa-general] Re: why is CPU util/service demand so much higher withSDP than TCP?

[Michael S. Tsirkin]

> Quoting Rick Jones <rick.jones2 at hp.com>:
> Subject: Re: [ofa-general] Re: why is CPU util/service demand so much higher withSDP than TCP?
> 
> Scott Weitzenkamp (sweitzen) wrote:
> ># modinfo ib_sdp
> >filename:
> >/lib/modules/2.6.16.21-0.8-smp/updates/kernel/drivers/infiniband
> >/ulp/sdp/ib_sdp.ko
> >author:         Michael S. Tsirkin
> >description:    InfiniBand SDP module
> >license:        Dual BSD/GPL
> >vermagic:       2.6.16.21-0.8-smp SMP gcc-4.1
> >depends:        ib_core,rdma_cm
> >srcversion:     91793E4825DEBC7A2DA9366
> >parm:           top_mem_usage:Top system wide sdp memory usage for recv
> >(in MB).
> > (int)
> >parm:           rcvbuf_scale:Receive buffer size scale factor. (int)
> >parm:           send_poll_thresh:Send message size thresh hold over
> >which to sta
> >rt polling. (int)
> >parm:           recv_poll:How many times to poll recv. (int)
> >parm:           send_poll:How many times to poll send. (int)
> >parm:           recv_poll_miss:How many times recv poll missed. (int)
> >parm:           recv_poll_hit:How many times recv poll helped. (int)
> >parm:           send_poll_miss:How many times send poll missed. (int)
> >parm:           send_poll_hit:How many times send poll helped. (int)
> >parm:           data_debug_level:Enable data path debug tracing if > 0.
> >(int)
> >parm:           debug_level:Enable debug tracing if > 0. (int)
> >
> 
> I've learned a new command today :)  And via other channels how to actually 
> see the current values of those things under /sys/modules/ib_sdp/parameters.
> 
> I think I may have also figured-out why my TCP_RR tests over IPoIB have 
> been consistently unable to hit confidence intervals for CPU utilization.  
> I am guessing that for IPoIB that path is effectively "recv_poll=0 and/or 
> send_poll=0" yes?

Right.

> When I set those values for ib_sdp and run the SDP_RR 
> test netperf is unable to be confident that the CPU util it measures is any 
> closer than +/- 20% of the "real" CPU utilization...  (still the OFED bits 
> in RHEL5 rather than 1.2)  With the defaults, the netperf SDP_RR test hit 
> the confidence intervals.

rick jones

-- 
MST