[openib-general] Timeline of IPoIB performance

[Roland Dreier]

    Rick> Which rev of netperf are you using, and areyou using the
    Rick> "confidence intervals" options (-i, -I)?  for a long time,
    Rick> the linux-unique behaviour of returning the overhead bytes
    Rick> for SO_[SND|RCV]BUF and them being 2X what one gives in
    Rick> setsockopt() gave netperf some trouble - the socket buffer
    Rick> would double in size each iteration on a confidence interval
    Rick> run.  Later netperf versions (late 2.3, and 2.4.X) have a
    Rick> kludge for this.

I believe it's netperf 2.2.

I'm not using any confidence interval stuff.  However, the variation
is not between single runs of netperf -- if I do 5 runs of netperf in
a row, I get roughly the same number from each run.  For example, I
might see something like

    TCP STREAM TEST to 192.168.145.2 : histogram
    Recv   Send    Send
    Socket Socket  Message  Elapsed
    Size   Size    Size     Time     Throughput
    bytes  bytes   bytes    secs.    10^6bits/sec
    
     87380  16384  16384    10.00    3869.82
    
and then

    TCP STREAM TEST to 192.168.145.2 : histogram
    Recv   Send    Send
    Socket Socket  Message  Elapsed
    Size   Size    Size     Time     Throughput
    bytes  bytes   bytes    secs.    10^6bits/sec
    
     87380  16384  16384    10.00    3862.41

for two successive runs.  However, if I reboot the system into the
same kernel (ie everything set up exactly the same), the same
invocation of netperf might give

    TCP STREAM TEST to 192.168.145.2 : histogram
    Recv   Send    Send
    Socket Socket  Message  Elapsed
    Size   Size    Size     Time     Throughput
    bytes  bytes   bytes    secs.    10^6bits/sec
    
     87380  16384  16384    10.00    4389.20

    Rick> Are there large changes in service demand along with the
    Rick> large performance changes?

Not sure.  How do I have netperf report service demand?

 - R.