[openib-general] Performance Degradation with OFED v. Voltaire

[Hal Rosenstock]

On Thu, 2006-12-14 at 09:39, Bernadat, Philippe wrote:
> So I did tried tune_pci=1. Didn't make any difference.
> 
> I used the same nodes to compare the lscpi output.
> I could see:
> 
> [root at axis21_EL4_u3 o2ib]$ diff lspci_axis19_vib lspci_axis21_ofed
> 30c30
> <       Capabilities: [40] MSI-X: Enable+ Mask- TabSize=32
> ---
> >       Capabilities: [40] MSI-X: Enable- Mask- TabSize=32

Might the MSI-X difference explain it ?

-- Hal

> 38,39c38,39
> < 40: 11 50 1f 80 00 20 08 00 00 22 08 00 00 00 00 00
> < 50: 03 60 ff ff 11 11 00 00 00 00 00 00 00 00 00 00
> ---
> > 40: 11 50 1f 00 00 20 08 00 00 22 08 00 00 00 00 00
> > 50: 03 60 ff 7f 11 11 00 00 00 00 00 00 00 00 00 00
> 
> So I added the ib_mthca msi_x=1 option.
> It didn't help.
> 
> So the only remaining difference now is:
> 
> [root at axis21_EL4_u3 o2ib]$ diff lspci_axis19_vib lspci_axis21_ofed 
> 39c39
> < 50: 03 60 ff ff 11 11 00 00 00 00 00 00 00 00 00 00
> ---
> > 50: 03 60 ff 7f 11 11 00 00 00 00 00 00 00 00 00 00
> 
> No idea what this is.
> 
> Philippe
> 
> > -----Original Message-----
> > From: Bernadat, Philippe 
> > Sent: Thursday, December 14, 2006 1:24 PM
> > To: Tziporet Koren
> > Cc: Eric Barton; Roland Dreier; Matt Leininger; 
> > openib-general at openib.org
> > Subject: RE: [openib-general] Performance Degradation with 
> > OFED v. Voltaire
> > 
> > 
> > > Have you tried running with
> > > 
> > > options ib_mthca tune_pci =1
> > > 
> > 
> > My understanding is that this is not required anymore with 
> > OFED-1.1 - It used to make a siginifciant differences with 
> > OFED-1.0, but I didn't observe it with OFED-1.1
> > 
> > And again, the user mode performance if comparable between 
> > VIB and OFED.
> > 
> > Philippe
> > 
> > > -----Original Message-----
> > > From: Tziporet Koren [mailto:tziporet at dev.mellanox.co.il] 
> > > Sent: Thursday, December 14, 2006 12:30 PM
> > > To: Bernadat, Philippe
> > > Cc: Eric Barton; Roland Dreier; Matt Leininger; 
> > > openib-general at openib.org; Bernadat, Philippe
> > > Subject: Re: [openib-general] Performance Degradation with 
> > > OFED v. Voltaire
> > > 
> > > Philippe Bernadat wrote:
> > > > Roland,
> > > >
> > > > Attached are the two lspci outputs.
> > > >
> > > > The only differences I see are:
> > > >
> > > > [philippe at hamish o2ib]$ diff lspci.vib lspci.ofed
> > > > 1d0
> > > > < pcilib: Resource 5 in 
> > > /sys/bus/pci/devices/0000:00:1f.1/resource has 
> > > > a 64-bit address, ignoring
> > > > 40c39
> > > > < 50: 03 60 ff 7f 11 11 00 00 00 00 00 00 00 00 00 00
> > > > ---
> > > > > 50: 03 60 ff ff 11 11 00 00 00 00 00 00 00 00 00 00
> > > > [philippe at hamish o2ib]$
> > > >
> > > Have you tried running with
> > > 
> > > options ib_mthca tune_pci =1
> > > 
> > > Tziporet
> > > 
> > > 
> 
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>