[ofa-general] ***SPAM*** OSU mpi latency > 100msec and > 1msec

[Rajouri Jammu]

Hi,
I modified/instrumented the OSU MP latency benchmark to measure time taken
by each transaction in order to get min and max latencies, in addition to
the average that's reported currently.
I noticed that some of the transactions, albeit few, took > 100usec and >
1msec.

Does anybody have any ideas about what could be causing such large round
trip times (>1msec) for a few transactions while the average looks pretty
good ( 10usec ranges) ?
Is it network or system issues?

Here is a snapshot of the output and attached is the modified code.

i'm using OFED 1.3, CentOS 5 and openmpi-1.2.5.

Any insights or ideas would very helpful. Thanks in advance.

Below shows # of transactions over 100usec and  1msec.
Iteration count was set to 60000 for each test.
Latency is round trip time.

--------------------------------------------------------------------------
# OSU MPI Latency Test v3.0
# Size            Latency (us)
0                        13.30   over_100usec: 13 over_1msec: 2 i 601000
1                        13.51   over_100usec: 12 over_1msec: 0 i 601000
2                        13.51   over_100usec: 13 over_1msec: 1 i 601000
4                        13.81   over_100usec: 42 over_1msec: 33 i 601000
8                        13.92   over_100usec: 36 over_1msec: 25 i 601000
16                       13.90   over_100usec: 10 over_1msec: 0 i 601000
32                       14.14   over_100usec: 54 over_1msec: 44 i 601000
64                       14.32   over_100usec: 11 over_1msec: 1 i 601000
128                      15.38   over_100usec: 10 over_1msec: 0 i 601000
256                      15.94   over_100usec: 14 over_1msec: 2 i 601000
512                      16.74   *over_100usec: 77 over_1msec: 65 *i 601000
1024                     21.07   over_100usec: 17 over_1msec: 0 i 601000
2048                     24.05   over_100usec: 17 over_1msec: 1 i 601000
4096                     29.99   over_100usec: 37 over_1msec: 5 i 601000
8192                     41.71   over_100usec: 39 over_1msec: 0 i 601000
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080808/d79d375f/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: osu_latency_profile.c
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20080808/d79d375f/attachment.c>