[ofa-general] ipoib performance (and xplot)

[chas williams - CONTRACTOR]

while looking at some ipoib performance i had a chance to graph the
tcp flow in xplot (see http://www.xplot.org/).  the graph appears very
strange and is attached to this message.

the lower solid line represent acks coming back from the tcp server, the
up line represent the window size (i disabled window scaling btw, this
doesnt affect performance).  the up/down arrows (they look like diamonds
due to scale) represent packets.  this is a view from the tcp client.

the initial part of the graph is tcp slow-start/congestion.  what is
curious to how the returning acks (after slow start is finished) seem
to get quiet periodically.  then the next ack that returns, then acks
the entire window.  this seems to be leading to a very bursty behavior.

i would normally expect to see two data packets followed by an
ack as can be seen between the 'burst' regions.

i see aboue 800Mb/s (good put) between the hosts which i understand
to get typical for ipoib.  there dont appear to be any link errors
either.

so why the long pauses?

ftp://ftp.cmf.nrl.navy.mil/pub/chas/ipoib.jpg