[openib-general] Performance Degradation with OFED v. Voltaire

[Bernadat, Philippe]

So I did tried tune_pci=1. Didn't make any difference.

I used the same nodes to compare the lscpi output.
I could see:

[root at axis21_EL4_u3 o2ib]$ diff lspci_axis19_vib lspci_axis21_ofed
30c30
<       Capabilities: [40] MSI-X: Enable+ Mask- TabSize=32
---
>       Capabilities: [40] MSI-X: Enable- Mask- TabSize=32
38,39c38,39
< 40: 11 50 1f 80 00 20 08 00 00 22 08 00 00 00 00 00
< 50: 03 60 ff ff 11 11 00 00 00 00 00 00 00 00 00 00
---
> 40: 11 50 1f 00 00 20 08 00 00 22 08 00 00 00 00 00
> 50: 03 60 ff 7f 11 11 00 00 00 00 00 00 00 00 00 00

So I added the ib_mthca msi_x=1 option.
It didn't help.

So the only remaining difference now is:

[root at axis21_EL4_u3 o2ib]$ diff lspci_axis19_vib lspci_axis21_ofed 
39c39
< 50: 03 60 ff ff 11 11 00 00 00 00 00 00 00 00 00 00
---
> 50: 03 60 ff 7f 11 11 00 00 00 00 00 00 00 00 00 00

No idea what this is.

Philippe

> -----Original Message-----
> From: Bernadat, Philippe 
> Sent: Thursday, December 14, 2006 1:24 PM
> To: Tziporet Koren
> Cc: Eric Barton; Roland Dreier; Matt Leininger; 
> openib-general at openib.org
> Subject: RE: [openib-general] Performance Degradation with 
> OFED v. Voltaire
> 
> 
> > Have you tried running with
> > 
> > options ib_mthca tune_pci =1
> > 
> 
> My understanding is that this is not required anymore with 
> OFED-1.1 - It used to make a siginifciant differences with 
> OFED-1.0, but I didn't observe it with OFED-1.1
> 
> And again, the user mode performance if comparable between 
> VIB and OFED.
> 
> Philippe
> 
> > -----Original Message-----
> > From: Tziporet Koren [mailto:tziporet at dev.mellanox.co.il] 
> > Sent: Thursday, December 14, 2006 12:30 PM
> > To: Bernadat, Philippe
> > Cc: Eric Barton; Roland Dreier; Matt Leininger; 
> > openib-general at openib.org; Bernadat, Philippe
> > Subject: Re: [openib-general] Performance Degradation with 
> > OFED v. Voltaire
> > 
> > Philippe Bernadat wrote:
> > > Roland,
> > >
> > > Attached are the two lspci outputs.
> > >
> > > The only differences I see are:
> > >
> > > [philippe at hamish o2ib]$ diff lspci.vib lspci.ofed
> > > 1d0
> > > < pcilib: Resource 5 in 
> > /sys/bus/pci/devices/0000:00:1f.1/resource has 
> > > a 64-bit address, ignoring
> > > 40c39
> > > < 50: 03 60 ff 7f 11 11 00 00 00 00 00 00 00 00 00 00
> > > ---
> > > > 50: 03 60 ff ff 11 11 00 00 00 00 00 00 00 00 00 00
> > > [philippe at hamish o2ib]$
> > >
> > Have you tried running with
> > 
> > options ib_mthca tune_pci =1
> > 
> > Tziporet
> > 
> >