[ofa-general] RE: [ewg] Not seeing any SDP performance changes inOFED 1.3 beta, and I get Oops when enabling sdp_zcopy_thresh

[Weikuan Yu]

Hi, Scott,

I have been running SDP tests across two woodcrest nodes with 4x DDR 
cards using OFED-1.2.5.4. The card/firmware info is below.

CA 'mthca0'
         CA type: MT25208
         Number of ports: 2
         Firmware version: 5.1.400
         Hardware version: a0
         Node GUID: 0x0002c90200228e0c
         System image GUID: 0x0002c90200228e0f

I could not get a bandwidth more than 5Gbps like you have shown here. 
Wonder if I need to upgrade to the latest software or firmware? Any 
suggestions?

Thanks,
--Weikuan


TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.225.77 
(192.168
.225.77) port 0 AF_INET
Recv   Send    Send                          Utilization       Service 
Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local 
remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

131072 131072 131072    10.00      4918.95   21.29    24.99    1.418 
1.665


Scott Weitzenkamp (sweitzen) wrote:
> Jim,
> 
> I am trying OFED-1.3-20071231-0600 and RHEL4 x86_64 on a dual CPU
> (single core each CPU) Xeon system.  I do not see any performance
> improvement (either throughput or CPU utilization) using netperf when I
> set /sys/module/ib_sdp/sdp_zcopy_thresh to 16384.  Can you elaborate on
> your HCA type, and performance improvement you see?
> 
> Here's an example netperf command line when using a Cheetah DDR HCA and
> 1.2.917 firmware (I have also tried ConnectX and 2.3.000 firmware too):
> 
> [releng at svbu-qa1850-2 ~]$ LD_PRELOAD=libsdp.so netperf241 -v2 -4 -H
> 192.168.1.201 -l 30 -t TCP_STREAM -c -C --   -m 65536
> TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.1.201
> (192.168.1.201) port 0 AF_INET : histogram : demo
> 
> Recv   Send    Send                          Utilization       Service
> Demand
> Socket Socket  Message  Elapsed              Send     Recv     Send
> Recv
> Size   Size    Size     Time     Throughput  local    remote   local
> remote
> bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB
> us/KB
> 
>  87380  16384  65536    30.01      7267.70   55.06    61.27    1.241
> 1.381 
> 
> Alignment      Offset         Bytes    Bytes       Sends   Bytes
> Recvs
> Local  Remote  Local  Remote  Xfered   Per                 Per
> Send   Recv    Send   Recv             Send (avg)          Recv (avg)
>     8       8      0       0 2.726e+10  65536.00    415942   48106.01
> 566648
>