[ofa-general] Re: 20% latency increase between UD to RC latency

[Or Gerlitz]

On Mon, 23 Jul 2007, Or Gerlitz wrote:

>> Running the latency test provided with the perftest package on my systems (*)
>> I get the below results. Does anyone has insight why the --minimal-- and typical
>> UD latency is 1us ( = 20%) worse then the --minimal-- and typical RC latency???

> running the latecy test on a similar system (RH5 smp / four way Xeon 1.9GHz /
> 4GB RAM) but this time with the memfree --Hermon-- HCA (25418 / FW 2.1.0) the minimal
> AND typical UD latecy is very much the same as the RC and UC ones which is ~1.5us
> So it both fixes the UD issue on Arbel and improves the latency from 4.5us to 1.5us

A third run, now over a memfree Sinai HCA (25204 / FW 1.2.0) the UD and RC latency are
quite the same, around 5.3us but the result is worse then the Arbel one in about 0.7us ...

Or.

[root at src1 ~]# ib_send_lat -c RC storm7
------------------------------------------------------------------
                    Send Latency Test
Inline data is used up to 400 bytes message
Connection type : RC
   local address: LID 0x09 QPN 0xd50407 PSN 0x7aaaf1
  remote address: LID 0x0b QPN 0x0405 PSN 0x292565
Mtu : 2048
------------------------------------------------------------------
 #bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]
      2        1000           5.27          63.27             5.32
------------------------------------------------------------------
[root at src1 ~]# ib_send_lat -c UD storm7
------------------------------------------------------------------
                    Send Latency Test
Inline data is used up to 400 bytes message
Connection type : UD
   local address: LID 0x09 QPN 0xd60407 PSN 0xcc70ba
  remote address: LID 0x0b QPN 0x10405 PSN 0x6794b5
------------------------------------------------------------------
 #bytes #iterations    t_min[usec]    t_max[usec]  t_typical[usec]
      2        1000           5.38          38.88             5.45
------------------------------------------------------------------