[openib-general] [PATCH] rewrite perftest/README

[Grant Grundler]

Michael,
Here's a complete rewrite of the README file.
Should make it easier for people to understand
	o how to build
	o run
	o interpret results


I'd still like to add abit more about statisical significance
of the sample size of 1000 but am just "refreshing my memory"
(ok, lame excuse, I've forgotten everything from 20years ago :^)
on how to do that. Maybe you can craft something based on your
experience plus the observations below?

Ditching the last two (extreme) readings from the "server" side
of the histogram:
	o standard deviation            86 cycles
	o "arithmetic mean" (average) 7135 cycles
	o median                      7126 cycles
        o min    6906 cycles (sorted sample #1)
	o max    7490 cycles (sorted sample #997)

(For the record, #998 is 8798 and #999 is 50305, we are clearly
measuring something else here too)

(1.5Ghz IA64, PCI-X, 2.6.11, forgot which SVN they are running)

The median value is *very* reproducible on this configuration.
+- 1 cycle consistently over 5 runs of rdma_lat.


thanks,
grant

Signed-off-by: Grant Grundler <iod00d at hp.com>


Index: perftest/README
===================================================================
--- perftest/README	(revision 2404)
+++ perftest/README	(working copy)
@@ -1,33 +1,84 @@
-This directory includes gen2 uverbs microbenchmarks -
-useful as usage examples and for performance tuning.
+This directory includes gen2 uverbs microbenchmarks.
 
-Testing methodology:
-- CPU clock instruction is used to get CPU clock without context switch.
-- Median (as opposed to average) result is reported. The median is less
-  sensitive to extreme scores. An option to report the full result
-  distribution for alternative statistical analysis is provided.
+The tests are intend as:
+	1) simple, efficient usage examples.
+		This code is GPL. Copy it literally only if you are
+		releasing GPL products. Otherwise use it as a guide only.
 
-Architectures supported:
-- i686, x86_64
+	2) a useful benchmark
+		e.g. for HW or SW tuning and/or functional testing.
+		Please post results/observations to openib-general email list.
+ 		See www.openib.org "Contact Us" link for details.
 
-Tests in this directory:
+
+Testing methodology
+-------------------
+
+- uses CPU cycle counter to get time stamps without context switch.
+  Some arches do NOT implement get_cycle(). e.g. Intel 80486 or older PPC.
+
+- measures round-trip time but reports half of that as one-way latency.
+  ie. May not be sufficiently accurate for asymetrical configurations.
+
+- Min/Median/Max result is always reported.
+  The median (vs average) is less sensitive to extreme scores.
+  Typically the "Max" value is the first value measured.
+
+- larger samples only marginally help. The default (1000) is pretty good.
+  Note that an array of cycles_t (typically unsigned long) is allocated
+  once to collect samples and again to store the difference between them.
+  Really big sample sizes (e.g. 1 million) might see other problems
+  with the program.
+
+- "-H" option will dump the histogram for additional statistical analysis.
+  See xgraph, ygraph, r-base (http://www.r-project.org/), pspp, or other 
+  statistical math programs.
+
+Architectures tested:	i686, x86_64, ia64
+
+
+
+Test Descriptions
+-----------------
 
 rdma_lat.c - latency test with RDMA write transactions
 
-Building:
+
+Build Tests
+-----------
 
 "make" to build all tests
 
-Running:
+	Debian: build-dep on linux-kernel-headers (for asm/timex.h file)
+		build-dep on libibverbs-dev
+		depends on libibverbs1
+
+
+Run Tests
+---------
+
+Prerequisites: 
+	kernel 2.6
+	ib_uverbs (kernel module) matches libibverbs
+		("match" means binary compatible, but ideally same SVN rev)
+	Debian: dpkg -i libibverbs1_0.1.0-1_ia64.deb
+
+Server:		./<test name> <options>
+Client:		./<test name> <options> <server IP address>
+
+	o IMPORTANT: The SAME OPTIONS must be passed to both server and client.
+	o "--help" will list the current <options>.
+	o <server address> is IPv4 or IPv6 address assigned to the IPoIB port.
 
-Server:
-	./<test name> <options>
-Client:
-	./<test name> <options> <server address>
+First Config IPoIB on both client and server with something like:
+	modprobe ib_mthca msi_x=1	# assumes platform supports MSI-X
+	modprobe ib_ipoib
+	ifconfig ib0 10.0.1.30 netmask 255.255.255.0
+	modprobe ib_uverbs
 
-Run
-	./<test name> --help
+Then (e.g.) "rdma_lat -UC | less" on the server side.
 
-For a list of options. Same options must be passed to both server and client.
+Run modprobe/ifconfig again on the client side (with a different IB).
+Lastly "rmda_lat -UC 10.0.1.31" on the client.
 
-MST
+rmda_lat will exit on both server and client after printing results.