So getting PCI config space access in domU will solve the problem ? if so how can I achieve that ?<br><br>~subbu<br><br><div class="gmail_quote">On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com">yunhong.jiang@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2">Sorry that seems the original mail has tried the permissive already
:$</font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2">How will So how will the card do the QEUREY_FW command?Through config
space or through MMIO? Following information is something strange, why all the
MMIO range is disabled?</font></span></div><div class="Ih2E3d">
<div dir="ltr" align="left"><span><font color="#0000ff" size="2"></font></span> </div>
<div dir="ltr" align="left"><span> Memory at fdc00000
(64-bit, non-prefetchable) [disabled] [size=1M]<br> Memory
at fd000000 (64-bit, prefetchable) [disabled] [size=8M]</span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2"></font></span> </div>
</div><div dir="ltr" align="left"><span><font color="#0000ff" size="2">As for the following information, I think it should be harmless since
domU has no method of config spacess access method.</font></span></div><div class="Ih2E3d">
<div> <font face="Courier New" size="2">PCI: Fatal: No PCI config space
access function found</font></div>
<div><font face="Courier New" size="2"></font> </div>
</div><div><span><font face="Courier New" size="2">Thanks</font></span></div>
<div><span><font face="Courier New" size="2">Yunhong
Jiang</font></span></div>
<div><br></div>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2"><div class="Ih2E3d"><b>From:</b> subbu kl [mailto:<a href="mailto:subbukl@gmail.com" target="_blank">subbukl@gmail.com</a>]
<br></div><b>Sent:</b> 2009年2月12日 14:43<div><div></div><div class="Wj3C7c"><br><b>To:</b> Jiang, Yunhong<br><b>Cc:</b>
David Brown; <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>;
<a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br><b>Subject:</b> Re: [Xen-devel] Re:
[ofa-general] Fwd: pciback module not working<br></div></div></font><br></div><div><div></div><div class="Wj3C7c">
<div></div>oops missed it,<br><br>well now I dont see that enable
permissive...message. here goes the messages what I got in dom0 while booting
domU <br><br>tap tap-1-51712: 2 getting info<br>pciback: vpci: 0000:0e:00.0:
assign to virtual slot 0<br>device vif1.0 entered promiscuous
mode<br>ADDRCONF(NETDEV_UP): vif1.0: link is not ready<br>blktap: ring-ref 9,
event-channel 9, protocol 1 (x86_64-abi)<br>PCI: Enabling device 0000:0e:00.0
(0000 -> 0002)<br>ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level,
low) -> IRQ 16<br>PCI: Setting latency timer of device 0000:0e:00.0 to
64<br>ACPI: PCI interrupt for device 0000:0e:00.0
disabled<br>ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready<br>xenbr0:
topology change detected, propagating<br>xenbr0: port 3(vif1.0) entering
forwarding state<br><br>any suspicious message ? <br>any Idea why I get that
:<br> <font face="Courier New" size="2">PCI: Fatal: No PCI config space
access function found<br>rtc: IRQ 8 is not free.<br><br><span style="font-family: arial,helvetica,sans-serif;">message in domU bootup message
?</span><br><br></font>~subbu<br><br>
<div class="gmail_quote">On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><font color="#0000ff"><font size="2"><span>So
a</span>ny<span> changes in dom0's dmesg?</span></font></font></div>
<div dir="ltr" align="left"><font color="#0000ff"><font size="2"><span></span></font></font> </div><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2"><b>From:</b> subbu kl [mailto:<a href="mailto:subbukl@gmail.com" target="_blank">subbukl@gmail.com</a>]
<br><b>Sent:</b> 2009年2月12日 13:52<br><b>To:</b> Jiang,
Yunhong<br><b>Cc:</b> David Brown; <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>; <a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br>
<b>Subject:</b> Re:
[Xen-devel] Re: [ofa-general] Fwd: pciback module not
working<br></font><br></div>
<div>
<div></div>
<div>
<div></div>
<div dir="ltr"><font color="#000000" face="Tahoma" size="2">no luck
!<br> dmesg in XEN PV guest shows :</font></div>
<div dir="ltr"><font face="Tahoma" size="2"></font> </div>
<div dir="ltr"><font face="Courier New" size="2">ib_mthca: Mellanox InfiniBand
HCA driver v1.0 (April 4, 2008)<br>ib_mthca: Initializing
0000:00:00.0<br>PCI: Enabling device 0000:00:00.0 (0000 ->
0002)<br>PCI: Setting latency timer of device 0000:00:00.0 to
64<br>ib_mthca 0000:00:00.0: QUERY_FW command failed,
aborting.<br>ib_mthca: probe of 0000:00:00.0 failed with error
-11<br><br></font></div>
<div dir="ltr">even after executingh the following in dom0:<br><br></div>
<div dir="ltr"><font face="Courier New" size="2">#echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/pciback/permissive<br><br></font></div>
<div dir="ltr">I am getting the follwing messages on the console as part of
the initial bootup messages of the guest:</div>
<div dir="ltr"> </div>
<div dir="ltr"><font face="Courier New" size="2">Started domain
rhel52_64_3<br>PCI: Fatal: No PCI config space access function
found<br>rtc: IRQ 8 is not free.<br>i8042.c: No controller
found.<br><br></font></div>
<div dir="ltr">after executing the following in dom0 :</div>
<div dir="ltr"><font face="Courier New" size="2">#xm create -c
rhel52_64_3<br><br><br><span style="font-family: verdana,sans-serif;">so,
problem persisits,</span></font></div>
<div dir="ltr"><font face="courier new" size="2"></font> </div>
<div dir="ltr">~subbu</div><br><br>
<div class="gmail_quote">2009/2/12 Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>></span><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2">Seems
it is because PCI frontend try to write some configuration space that
PCIback has no config_field entry to support it.</font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2">I think
you can firstly try to do as dom0's dmesg suggested: "<font color="#000000" face="Times New Roman" size="3">see permissive attribute in
sysfs" (it should be "set permissive attribute...", I
think).</font></font></span></div>
<div dir="ltr" align="left"><span></span> </div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2">BTW, where you got following log? That seems suggest config
space function not found.</font></span></div>
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2"><font color="#000000" face="Times New Roman" size="3"><font color="#0000ff" face="宋体" size="2"></font><br>PCI: Fatal: No PCI config space access function
found<br>rtc: IRQ 8 is not free.<br>i8042.c: No controller
found."</font></font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2"></font></span> </div><span></span></div><font face="Times New Roman">-- Yunhong Jiang<span></span></font><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2"><b>From:</b> <a href="mailto:xen-devel-bounces@lists.xensource.com" target="_blank">xen-devel-bounces@lists.xensource.com</a> [mailto:<a href="mailto:xen-devel-bounces@lists.xensource.com" target="_blank">xen-devel-bounces@lists.xensource.com</a>] <b>On Behalf
Of </b>subbu kl<br><b>Sent:</b> 2009年2月11日 22:18<br><b>To:</b> David
Brown<br><b>Cc:</b> <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>; <a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br>
<b>Subject:</b>
[Xen-devel] Re: [ofa-general] Fwd: pciback module not
working<br></font><br></div>
<div>
<div></div>
<div>
<div></div>I am getting the same QUERY_FW failed on RHEL5.2 with
xenxen paravirtualized guest with pciback module.<br><br>No one seems
to have tried answering this question on the list, let me ping
xen-devel and ofed people again.<br><br>after executing in
dom0<br>echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/ib_mthca/unbind<br>echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/pciback/new_slot<br>echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/pciback/bind<br><br>#dmesg<br>ACPI: PCI interrupt
for device 0000:0e:00.0 disabled<br>tap tap-1-51712: 2 getting
info<br>tap tap-2-51712: 2 getting info<br>pciback 0000:0e:00.0:
seizing device<br>PCI: Enabling device 0000:0e:00.0 (0140 ->
0142)<br>ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low)
-> IRQ 16<br>ACPI: PCI interrupt for device 0000:0e:00.0
disabled<br><br>#xm create -c rhel52_64_3<br><br>PCI: Fatal: No PCI
config space access function found<br>rtc: IRQ 8 is not
free.<br>i8042.c: No controller found.<br><br><br>GUEST
dmesg:<br><br>ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April 4,
2008)<br>ib_mthca: Initializing 0000:00:00.0<br>PCI: Enabling device
0000:00:00.0 (0000 -> 0002)<br>PCI: Setting latency timer of device
0000:00:00.0 to 64<br>ib_mthca 0000:00:00.0: QUERY_FW command failed,
aborting.<br>ib_mthca: probe of 0000:00:00.0 failed with error
-11<br><br>in dom0:<br>Feb 11 19:44:37 p128 kernel: tap tap-3-51712: 2
getting info<br>Feb 11 19:44:37 p128 kernel: pciback: vpci:
0000:0e:00.0: assign to virtual slot 0<br>Feb 11 19:44:37 p128 kernel:
device vif3.0 entered promiscuous mode<br>Feb 11 19:44:37 p128 kernel:
ADDRCONF(NETDEV_UP): vif3.0: link is not ready<br>Feb 11 19:44:39 p128
kernel: blktap: ring-ref 9, event-channel 9, protocol 1
(x86_64-abi)<br>Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0:
Driver tried to write to a read-only configuration space field at
offset 0x44, size 2. This may be harmless, but if you have problems
with your device:<br>Feb 11 19:44:48 p128 kernel: 1) see permissive
attribute in sysfs<br>Feb 11 19:44:48 p128 kernel: 2) report problems
to the xen-devel mailing list along with details of your device
obtained from lspci.<br>Feb 11 19:44:48 p128 kernel: PCI: Enabling
device 0000:0e:00.0 (0000 -> 0002)<br>Feb 11 19:44:48 p128 kernel:
ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low) ->
IRQ 16<br>Feb 11 19:44:49 p128 kernel: ACPI: PCI interrupt for device
0000:0e:00.0 disabled<br><br><br><br>some more details - [root@p128
~]# rpm -qa | grep
xen<br>kernel-xen-2.6.18-92.1.22.el5<br>xen-3.0.3-64.el5_2.9<br>xen-libs-3.0.3-64.el5_2.9<br>xen-libs-3.0.3-64.el5_2.9<br><br>[root@p128
~]# ibv_devinfo<br>hca_id:
mthca0<br>
fw_ver:
5.3.0<br>
node_guid:
0002:c902:0022:cd48<br>
sys_image_guid:
0002:c902:0022:cd4b<br>
vendor_id:
0x02c9<br>
vendor_part_id:
25218<br>
hw_ver:
0x20<br>
board_id:
MT_0370130002<br>
phys_port_cnt:
2<br>
port:
1<br>
state:
PORT_INIT
(2)<br>
max_mtu:
2048
(4)<br>
active_mtu:
512
(2)<br>
sm_lid:
0<br>
port_lid:
0<br>
port_lmc:
0x00<br><br>
port:
2<br>
state:
PORT_DOWN
(1)<br>
max_mtu:
2048
(4)<br>
active_mtu:
512
(2)<br>
sm_lid:
0<br>
port_lid:
0<br>
port_lmc:
0x00<br><br><br>any help greatly appreciated.<br><br>~subbu<br><br>
<div class="gmail_quote">On Sat, Oct 18, 2008 at 4:54 AM, David Brown
<span dir="ltr"><<a href="mailto:dmlb2000@gmail.com" target="_blank">dmlb2000@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Okay
so my question to the openfabrics guys is, why would the
OFED<br>drivers fail to read the firmware?<br><br>Any
thoughts?<br><br>Thanks,<br>- David Brown<br><br><br>----------
Forwarded message ----------<br>From: David Brown <<a href="mailto:dmlb2000@gmail.com" target="_blank">dmlb2000@gmail.com</a>><br>Date: Thu, Sep 11, 2008
at 2:24 PM<br>Subject: pciback module not working<br>To: <a href="mailto:xen-users@lists.xensource.com" target="_blank">xen-users@lists.xensource.com</a>, <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a><br>
<br><br>This
issue was brought up about a year and a half ago. So I'll
bring<br>it up again and see if anything happens.<br><br>I've got an
infiniband network and am attempting to pass the<br>infiniband card
through the host and give it to the guest.<br>I'm working with
standard CentOS 5.2 on both guest and host with their<br>provided
xen (3.0.3 ish). I've also attempted to install the newest<br>Xen
3.3 and use their standard host kernel and that did the
same<br>thing. The guest dmesg output in the guest is similar on
both<br>permissive and normal mode.<br><br>I'm getting issues with
detecting the firmware on the card for some reason...<br><br>Any
help would be appreciated.<br><br>Thanks,<br>- David
Brown<br><br>=== GUEST dmesg output ===<br>ib_mthca: Mellanox
InfiniBand HCA driver v1.0 (February 28, 2008)<br>ib_mthca:
Initializing 0000:00:00.0<br>PCI: Enabling device 0000:00:00.0 (0000
-> 0002)<br>PCI: Setting latency timer of device 0000:00:00.0 to
64<br>ib_mthca 0000:00:00.0: QUERY_FW command failed,
aborting.<br>ib_mthca: probe of 0000:00:00.0 failed with error
-11<br>=======================<br><br>=== Host modprobe.conf
===<br>alias eth0 bnx2<br>alias eth1 bnx2<br>alias scsi_hostadapter
cciss<br>options pciback
hide=(41:00.0)<br>=====================<br><br>=== Host lspci output
===<br># lspci -vs 41:00.0<br>41:00.0 InfiniBand: Mellanox
Technologies MT25204 [InfiniHost III Lx<br>HCA] (rev 20)<br>
Subsystem: Hewlett-Packard Company Unknown device
170a<br> Flags: fast devsel, IRQ 16<br>
Memory at fdc00000 (64-bit, non-prefetchable)
[disabled] [size=1M]<br> Memory at fd000000
(64-bit, prefetchable) [disabled] [size=8M]<br>
Capabilities: [40] Power Management version 2<br>
Capabilities: [48] Vital Product Data<br>
Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5
Enable-<br> Capabilities: [84] MSI-X: Enable-
Mask- TabSize=32<br> Capabilities: [60] Express
Endpoint IRQ 0<br>=====================<br><br>This makes sure it
get loaded first off before anything else.<br>=== Host mkinitrd cmd
===<br># mkinitrd -f --with=pciback --preload
pciback<br>/boot/initrd-2.6.18-92.1.10.el5xen.img
2.6.18-92.1.10.el5xen<br>====================<br><br>=== Host
pciback dmesg ===<br>pciback 0000:41:00.0: Driver tried to write to
a read-only<br>configuration space field at offset 0x44, size 2.
This may be<br>harmless, but if you have problems with your
device:<br>1) see permissive attribute in sysfs<br>2) report
problems to the xen-devel mailing list along with details of<br>your
device obtained from lspci.<br>PCI: Enabling device 0000:41:00.0
(0000 -> 0002)<br>ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI
16 (level, low) -> IRQ 16<br>PCI: Setting latency timer of device
0000:41:00.0 to 64<br>ACPI: PCI interrupt for device 0000:41:00.0
disabled<br>======================<br><br>=== Host pciback dmesg
(after setting it permissive) ===<br>pciback 0000:41:00.0: enabling
permissive mode configuration space accesses!<br>pciback
0000:41:00.0: permissive mode is potentially unsafe!<br>pciback:
vpci: 0000:41:00.0: assign to virtual slot 0<br>device vif1.0
entered promiscuous mode<br>ADDRCONF(NETDEV_UP): vif1.0: link is not
ready<br>blkback: ring-ref 9, event-channel 28, protocol 1
(x86_64-abi)<br>PCI: Enabling device 0000:41:00.0 (0000 ->
0002)<br>ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16 (level,
low) -> IRQ 16<br>PCI: Setting latency timer of device
0000:41:00.0 to 64<br>ACPI: PCI interrupt for device 0000:41:00.0
disabled<br>=========================================<br><br>===
Guest lspci output ===<br># lspci -v<br>00:00.0 InfiniBand: Mellanox
Technologies MT25204 [InfiniHost III Lx<br>HCA] (rev 20)<br>
Subsystem: Hewlett-Packard Company Unknown device
170a<br> Flags: fast devsel, IRQ 16<br>
Memory at fdc00000 (64-bit, non-prefetchable)
[disabled] [size=1M]<br> Memory at fd000000
(64-bit, prefetchable) [disabled] [size=8M]<br>
Capabilities: [40] Power Management version 2<br>
Capabilities: [48] Vital Product Data<br>
Capabilities: [90] Message Signalled Interrupts: 64bit+<br>Queue=0/5
Enable-<br> Capabilities: [84] MSI-X: Enable-
Mask- TabSize=32<br> Capabilities: [60] Express
Endpoint IRQ
0<br>=====================<br>_______________________________________________<br>general
mailing list<br><a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br><a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" target="_blank">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br>
<br>To
unsubscribe, please visit <a href="http://openib.org/mailman/listinfo/openib-general" target="_blank">http://openib.org/mailman/listinfo/openib-general</a><br></blockquote></div><br><br clear="all"><br>-- <br>
. . . s u b b u<br>"You've got to be original,
because if you're like someone else, what do they need you
for?"<br></div></div></blockquote></div></blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original,
because if you're like someone else, what do they need you
for?"<br></div></div></blockquote></div></blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original, because if
you're like someone else, what do they need you
for?"<br></div></div></blockquote></div>
</blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original, because if you're like someone else, what do they need you for?"<br>