[ewg] Re: the default CQ moderation parameters patch in OFED 1.3

[Eli Cohen]

One more thing regarding latency - you can use ethtool to modify these
parameters if your application is latency sensitive.

On Sun, 2008-01-06 at 10:59 +0200, Eli Cohen wrote:
> On Sun, 2008-01-06 at 10:47 +0200, Or Gerlitz wrote:
> > Hi Eli,
> > 
> > The patch below in OFED 1.3 does not check the device capabilities and hence
> > always fail on non connectx systems. Can you fix it such that we will not
> > get all those "why I we mthca0: failed to modify CQ params prints in the logs".
> I chose to use KERN_INFO to indicate that it is not a sever situation.
> Perhaps I should just remove the message.
> > 
> > Other then that and maybe even more important... I understand that it hard codes
> > ipoib to ask for delivery of interrupt only after MAX (16 packets received, 10 us
> > elapsed since first packet received), correct? so every simple ping-pong test that
> > measures IPoIB latency under small packet rate will have now 10us added to its latency?
> I did not notice that this deteriorates ping pong latency. Could you
> check if it does?
> 
> > 
> > I wonder if we can come up with simple way-in-the-middle here.
> > 
> > thanks,
> > 
> > Or.
> > ------------------------------------
> > Set default CQ moderation parameters
> > 
> > Set the default params to make sure they are applied. Otherwise
> > we get poor performance.
> > 
> > Signed-off-by: Eli Cohen <eli at mellanox.co.il>
> > 
> > Index: ofa_kernel-1.3/drivers/infiniband/ulp/ipoib/ipoib_verbs.c
> > ===================================================================
> > --- ofa_kernel-1.3.orig/drivers/infiniband/ulp/ipoib/ipoib_verbs.c	2007-11-19 18:05:29.000000000 +0200
> > +++ ofa_kernel-1.3/drivers/infiniband/ulp/ipoib/ipoib_verbs.c	2007-11-19 18:06:52.000000000 +0200
> > @@ -181,6 +181,9 @@
> >  		goto out_free_mr;
> >  	}
> > 
> > +	if (ib_modify_cq(priv->cq, 16, 10))
> > +		printk(KERN_INFO "%s: failed to modify CQ params\n", ca->name);
> > +
> >  	if (ib_req_notify_cq(priv->cq, IB_CQ_NEXT_COMP))
> >  		goto out_free_cq;
> >