[openib-general] perftest on i386

[Grant Grundler]

On Fri, May 20, 2005 at 04:59:44PM +0200, Bernhard Fischer wrote:
> Hi,
> 
> On i386 UP, i see odd results during runs. The machine is basically
> idle.

The runs are really short if you are using the default (-n 1000).
I don't expect them to be visible to something like top or vmstat.

> The worst latency is always somewhere around exchange 12-80.
> Is this expected and if not, does somebody have an idea what may cause
> this?

It probably depends on which x86 chipset you are using and what
else might be going on.

> Thank you,
> 
> $ ./rdma_lat -i 2 -U -s 1  tst09 -t 1 | egrep -v ", 3.(1|2|3)"
>   local address:  LID 0x02, QPN 0x1a0406, PSN 0x5379e1 RKey 0x1040436
> VAddr 0x00000008051001
>   remote address: LID 0x01, QPN 0x1d0406, PSN 0xd7aed6, RKey 0x188043c
> VAddr 0x00000008051001
> #, usec
> 1, 12.8368

Interesting that this is only ~4x normal latency.

> 2, 4.94677
> 12, 7.0644
> 13, 87.8933
> 26, 64.8241

Could it be we are crossing a cacheline boundary someplace
every 13 or 14 exchanges?

Any simple statistical method to determine if the long
delays are cyclical?

grant