[ofa-general] performance drop for datagram mode with the newconnectx FW

[Suresh Shelvapille]

I also get about 650MB for datagram mode with 2.3 FW.
 
> -----Original Message-----
> From: general-bounces at lists.openfabrics.org [mailto:general-bounces at lists.openfabrics.org] On Behalf
> Of Eli Cohen
> Sent: Monday, June 23, 2008 7:43 AM
> To: Or Gerlitz
> Cc: Eli Cohen; Roland Dreier; general at lists.openfabrics.org
> Subject: Re: [ofa-general] performance drop for datagram mode with the newconnectx FW
> 
> On Mon, Jun 23, 2008 at 02:36:39PM +0300, Or Gerlitz wrote:
> > Eli,
> >
> > Using the new connectx FW (2.5), I see performance drop to almost
> > zero with ipoib datagram mode. The code that runs on these systems
> > is ofed 1.3 and not mainline kernel, details below.
> >
> > Running netperf With connected mode (64k MTU) I get about 950MB/s
> > where with datagram mode (2k MTU) I get only 20-40MB/s. I used to
> > see about 650MB/s and above with FW 2.3 and datagram mode. Not that
> > it could explain the drop, but the NIC reports to the OS stateless
> > offload support - /sys/class/net/ib1/features is 0x11423
> >
> > I have opened the ipoib and mlx4 debug prints, and I don't see anything
> > special other then the dmesg get quite filled with
> >
> > 	ib1: TX ring full, stopping kernel net queue
> >
> > any idea what can explain this? ibv_ud_pingpong gives about 2Gb/s which
> > is about five times what I see with ipoib.
> >
> >
> > Or.
> >
> > git://git.openfabrics.org/ofed_1_3/linux-2.6.git ofed_kernel
> > commit 564e9e9383272f4311fd87ff4e5447cfcebad73a
> >
> > # uname -a
> > Linux gen2-1 2.6.18-53.el5 #1 SMP Wed Oct 10 16:34:19 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
> >
> > # cat /etc/redhat-release
> > Red Hat Enterprise Linux Server release 5.1 (Tikanga)
> 
> Can you tell if changing the FW to 2.3 gives more reasonable results?
> I don't believe such a drop in performance would have passed the QA
> tests but I'll check that.
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general