[ofa-general] Re: performance drop for datagram mode with the new connectx FW

[Or Gerlitz]

some details on the setup / nodes

I used the fw-25408-2_5_000-MHGH28-XTC_A1.bin (size 487388) firmware file
which was downloaded from the mellanox website and this command

	$ mstflint -d 07:00.0 -i /tmp/fw-25408-2_5_000-MHGH28-XTC_A1.bin b

to burn it (eg on a node with the hca at position 07:00.0)


The connectx HCAs we use here are eagle-DDR, I can double check it if this matters

node A (internal IP 172.25.5.157)
=================================
- RH 5.1 kernel 2.6.18-53.el5 SMP x86_84
- 8GB RAM
- 2 quad core 2.6 GHZ Intel CPUs

$ lspci | grep IB

07:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX IB DDR] (rev a0)

$ mstflint -d 07:00.0 q

Image type:      ConnectX
FW Version:      2.5.0
Device ID:       25418
Chip Revision:   A0
Description:     Node             Port1            Port2            Sys
image
GUIDs:           0002c90200258630 0002c90200258631 0002c90200258632 0002c90200258633
Board ID:         (MT_04A0110002)
VSD:
PSID:            MT_04A0110002

node B (internal IP 172.25.5.77)
=================================
- RH 5  kernel 2.6.18-8.el5 SMP x86_84
- 8GB RAM
- 2 dual core 2.6 GHZ Intel CPUs

$ lspci | grep Inf
03:00.0 InfiniBand: Mellanox Technologies Unknown device 634a (rev a0)

$ mstflint -d 03:00.0 q
Image type:      ConnectX
FW Version:      2.5.0
Device ID:       25418
Chip Revision:   A0
Description:     Node             Port1            Port2            Sys
image
GUIDs:           0002c9020025821c 0002c9020025821d 0002c9020025821e 0002c9020025821f
Board ID:         (MT_04A0110002)
VSD:
PSID:            MT_04A0110002