[openib-general] FW: Mstflint - not working on ppc64 and whendriver is not loaded on AMD

[Tseng-Hui (Frank) Lin]

On Wed, 2006-09-27 at 18:19 +0300, Michael S. Tsirkin wrote:
> Quoting r. Moshe Kazir <moshek at voltaire.com>:
> > Subject: FW: [openib-general] Mstflint - not working on ppc64 and whendriver is not loaded on AMD
> > 
> > Michael,
> >  
> > Frank new version was tested once more in Voltaire and is working o.k. .
> > I tested  `./mstflint -d <lspci output> q`  when drivers are loaded and when drivers are not loaded. in all cases it worked o.k.
> 
> Thanks for testing, but I'd like to get a handle on what's going on first.
> 
> First, I'm pretty sure when driver is loaded things work OK on all systems.
> When driver is not loaded - could you please answer whether using
> /sys/bus/pci/devices/0000\:03\:00.0/resource0
> works for you (on systems that have resource0)?
> 

It doesn't work.

> >  
> > Test was ferformed on the following environments :
> >  
> > -    IBM js21 ppc64 sles10 PCI-E
> > -    IBM js21 ppc64 sles9 sp3 PCI-E
> > -    IBM hs21 em64t redhat as 4 u3 PCI-E
> > -    IBM hs21 em64t sles 9 sp3 PCI-E
> > -    x86_64 sles10  PCI-E
> > -    MAC ppc64 sles10 PCI-X
> > -    MAC ppc64 sles10 PCI-E
> >
> > Please consider inserting the patch to OFED .
> >  
> > Moshe
> 
> Since I don't consider this a critical fix (there's no reason driver won't go
> up, and if it does not, there's a simple workaround by specifying the /proc
> interface, that is slower but works), I don't think this should go into OFED 1.1.
> 
> Unfortunately, I never got a small bugfix patch against the latest mstflint -
> the patch I saw posted touches all kind of things all over the code -
> so I can't insert it in trunk, either.
> 

I agree this is not critical. The patch changes nothing but the way of
opening the device.

On some ppc64 and x86_64 machines, the I/O memory mapped by mmap() is
not accessable (return 0xFFFFFFFF) unless the kernel code (usually the
device driver) does an ioremap. This is why mmap resource0 does not work
on these machines. There is no way I am aware of can do ioremap from
user space code like mstflint. The only thing I can think of is to fall
back to use the config space file in /proc/bus/pci/.

The (big) patch I made checks if the faster way (mmap resource0) works.
It it doesn't, the patch tries other slower ways and use the fastest
working way it can find. That's all the patch does. It does not make big
fix. It just save the users trouble of trying all possible ways of
opening a devices manually.

I understand applying big patch is risky unless it can be throughly
tested. Unfortunately, no one has all the machines to test the patch.
Moshe and I have tested the patch on Power MAC, Squadrons, JS20, and
JS21 (almost all living ppc64 machines) as well as a few x86_64
machines. We believe this patch is safe for these machines. The patch
can be enabled by defining CONFIG_MOPEN_FALL_BACK to 1.
CONFIG_MOPEN_FALL_BACK is defined to 1 for ppc64 and x86_64 and 0 for
others. We can enable this patch on other machines when people who have
these machines tested the patch.

I agree this is no a critical patch, but it is a useful one. Moreover,
it is well tested on the machines with the patch enabled and change
nothing on the machines with the patch disabled. I believe this is a
safe patch. Please re-consider adding it. Thanks.