so back to square one ? <br>Why QUERY_FW should fail in domU ?<br><br>~subbu<br><br><div class="gmail_quote">On Thu, Feb 12, 2009 at 12:30 PM, Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com">yunhong.jiang@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><font color="#0000ff" size="2"><span>DomU access config space through pcibackend, so that
message is ok.</span></font></div><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2"><div class="Ih2E3d"><b>From:</b> subbu kl [mailto:<a href="mailto:subbukl@gmail.com" target="_blank">subbukl@gmail.com</a>]
<br></div><b>Sent:</b> 2009年2月12日 14:59<div><div></div><div class="Wj3C7c"><br><b>To:</b> Jiang, Yunhong<br><b>Cc:</b>
David Brown; <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>;
<a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br><b>Subject:</b> Re: [Xen-devel] Re:
[ofa-general] Fwd: pciback module not working<br></div></div></font><br></div><div><div></div><div class="Wj3C7c">
<div></div>So getting PCI config space access in domU will solve the problem ?
if so how can I achieve that ?<br><br>~subbu<br><br>
<div class="gmail_quote">On Thu, Feb 12, 2009 at 12:26 PM, Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2">Sorry that seems
the original mail has tried the permissive already :$</font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2">How will So how
will the card do the QEUREY_FW command?Through config space or through MMIO?
Following information is something strange, why all the MMIO range is
disabled?</font></span></div>
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2"></font></span> </div>
<div dir="ltr" align="left"><span> Memory at
fdc00000 (64-bit, non-prefetchable) [disabled] [size=1M]<br>
Memory at fd000000 (64-bit, prefetchable) [disabled]
[size=8M]</span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2"></font></span> </div></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2">As for the
following information, I think it should be harmless since domU has no
method of config spacess access method.</font></span></div>
<div>
<div> <font face="Courier New" size="2">PCI: Fatal: No PCI config space
access function found</font></div>
<div><font face="Courier New" size="2"></font> </div></div>
<div><span><font face="Courier New" size="2">Thanks</font></span></div>
<div><span><font face="Courier New" size="2">Yunhong Jiang</font></span></div>
<div><br></div>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2">
<div><b>From:</b> subbu kl [mailto:<a href="mailto:subbukl@gmail.com" target="_blank">subbukl@gmail.com</a>]
<br></div><b>Sent:</b> 2009年2月12日 14:43
<div>
<div></div>
<div><br><b>To:</b> Jiang, Yunhong<br><b>Cc:</b> David Brown;
<a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>; <a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br><b>Subject:</b> Re:
[Xen-devel] Re: [ofa-general] Fwd: pciback module not
working<br></div></div></font><br></div>
<div>
<div></div>
<div>
<div></div>oops missed it,<br><br>well now I dont see that enable
permissive...message. here goes the messages what I got in dom0 while
booting domU <br><br>tap tap-1-51712: 2 getting info<br>pciback: vpci:
0000:0e:00.0: assign to virtual slot 0<br>device vif1.0 entered
promiscuous mode<br>ADDRCONF(NETDEV_UP): vif1.0: link is not
ready<br>blktap: ring-ref 9, event-channel 9, protocol 1
(x86_64-abi)<br>PCI: Enabling device 0000:0e:00.0 (0000 ->
0002)<br>ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI 16 (level, low)
-> IRQ 16<br>PCI: Setting latency timer of device 0000:0e:00.0 to
64<br>ACPI: PCI interrupt for device 0000:0e:00.0
disabled<br>ADDRCONF(NETDEV_CHANGE): vif1.0: link becomes ready<br>xenbr0:
topology change detected, propagating<br>xenbr0: port 3(vif1.0) entering
forwarding state<br><br>any suspicious message ? <br>any Idea why I get
that :<br> <font face="Courier New" size="2">PCI: Fatal: No PCI config
space access function found<br>rtc: IRQ 8 is not free.<br><br><span style="font-family: arial,helvetica,sans-serif;">message in domU bootup
message ?</span><br><br></font>~subbu<br><br>
<div class="gmail_quote">On Thu, Feb 12, 2009 at 11:50 AM, Jiang, Yunhong
<span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><font color="#0000ff"><font size="2"><span>So
a</span>ny<span> changes in dom0's dmesg?</span></font></font></div>
<div dir="ltr" align="left"><font color="#0000ff"><font size="2"><span></span></font></font> </div><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2"><b>From:</b> subbu kl [mailto:<a href="mailto:subbukl@gmail.com" target="_blank">subbukl@gmail.com</a>]
<br><b>Sent:</b> 2009年2月12日 13:52<br><b>To:</b> Jiang,
Yunhong<br><b>Cc:</b> David Brown; <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>; <a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br>
<b>Subject:</b> Re:
[Xen-devel] Re: [ofa-general] Fwd: pciback module not
working<br></font><br></div>
<div>
<div></div>
<div>
<div></div>
<div dir="ltr"><font color="#000000" face="Tahoma" size="2">no luck
!<br> dmesg in XEN PV guest shows :</font></div>
<div dir="ltr"><font face="Tahoma" size="2"></font> </div>
<div dir="ltr"><font face="Courier New" size="2">ib_mthca: Mellanox
InfiniBand HCA driver v1.0 (April 4, 2008)<br>ib_mthca: Initializing
0000:00:00.0<br>PCI: Enabling device 0000:00:00.0 (0000 ->
0002)<br>PCI: Setting latency timer of device 0000:00:00.0 to
64<br>ib_mthca 0000:00:00.0: QUERY_FW command failed,
aborting.<br>ib_mthca: probe of 0000:00:00.0 failed with error
-11<br><br></font></div>
<div dir="ltr">even after executingh the following in
dom0:<br><br></div>
<div dir="ltr"><font face="Courier New" size="2">#echo -n 0000:0e:00.0
> /sys/bus/pci/drivers/pciback/permissive<br><br></font></div>
<div dir="ltr">I am getting the follwing messages on the console as part
of the initial bootup messages of the guest:</div>
<div dir="ltr"> </div>
<div dir="ltr"><font face="Courier New" size="2">Started domain
rhel52_64_3<br>PCI: Fatal: No PCI config space access function
found<br>rtc: IRQ 8 is not free.<br>i8042.c: No controller
found.<br><br></font></div>
<div dir="ltr">after executing the following in dom0 :</div>
<div dir="ltr"><font face="Courier New" size="2">#xm create -c
rhel52_64_3<br><br><br><span style="font-family: verdana,sans-serif;">so, problem
persisits,</span></font></div>
<div dir="ltr"><font face="courier new" size="2"></font> </div>
<div dir="ltr">~subbu</div><br><br>
<div class="gmail_quote">2009/2/12 Jiang, Yunhong <span dir="ltr"><<a href="mailto:yunhong.jiang@intel.com" target="_blank">yunhong.jiang@intel.com</a>></span><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2">Seems it is because PCI frontend try to write some
configuration space that PCIback has no config_field entry to
support it.</font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2">I
think you can firstly try to do as dom0's dmesg suggested: "<font color="#000000" face="Times New Roman" size="3">see permissive attribute
in sysfs" (it should be "set permissive attribute...", I
think).</font></font></span></div>
<div dir="ltr" align="left"><span></span> </div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2">BTW, where you got following log? That seems suggest
config space function not found.</font></span></div>
<div>
<div dir="ltr" align="left"><span><font color="#0000ff" face="宋体" size="2"><font color="#000000" face="Times New Roman" size="3"><font color="#0000ff" face="宋体" size="2"></font><br>PCI: Fatal: No PCI config
space access function found<br>rtc: IRQ 8 is not free.<br>i8042.c:
No controller found."</font></font></span></div>
<div dir="ltr" align="left"><span><font color="#0000ff" size="2"></font></span> </div><span></span></div><font face="Times New Roman">-- Yunhong Jiang<span></span></font><br>
<blockquote style="border-left: 2px solid rgb(0, 0, 255); padding-left: 5px; margin-left: 5px; margin-right: 0px;">
<div dir="ltr" align="left" lang="en-us">
<hr>
<font face="Tahoma" size="2"><b>From:</b> <a href="mailto:xen-devel-bounces@lists.xensource.com" target="_blank">xen-devel-bounces@lists.xensource.com</a> [mailto:<a href="mailto:xen-devel-bounces@lists.xensource.com" target="_blank">xen-devel-bounces@lists.xensource.com</a>] <b>On
Behalf Of </b>subbu kl<br><b>Sent:</b> 2009年2月11日
22:18<br><b>To:</b> David Brown<br><b>Cc:</b> <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a>; <a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br>
<b>Subject:</b>
[Xen-devel] Re: [ofa-general] Fwd: pciback module not
working<br></font><br></div>
<div>
<div></div>
<div>
<div></div>I am getting the same QUERY_FW failed on RHEL5.2 with
xenxen paravirtualized guest with pciback module.<br><br>No one
seems to have tried answering this question on the list, let me
ping xen-devel and ofed people again.<br><br>after executing in
dom0<br>echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/ib_mthca/unbind<br>echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/pciback/new_slot<br>echo -n 0000:0e:00.0 >
/sys/bus/pci/drivers/pciback/bind<br><br>#dmesg<br>ACPI: PCI
interrupt for device 0000:0e:00.0 disabled<br>tap tap-1-51712: 2
getting info<br>tap tap-2-51712: 2 getting info<br>pciback
0000:0e:00.0: seizing device<br>PCI: Enabling device 0000:0e:00.0
(0140 -> 0142)<br>ACPI: PCI Interrupt 0000:0e:00.0[A] -> GSI
16 (level, low) -> IRQ 16<br>ACPI: PCI interrupt for device
0000:0e:00.0 disabled<br><br>#xm create -c rhel52_64_3<br><br>PCI:
Fatal: No PCI config space access function found<br>rtc: IRQ 8 is
not free.<br>i8042.c: No controller found.<br><br><br>GUEST
dmesg:<br><br>ib_mthca: Mellanox InfiniBand HCA driver v1.0 (April
4, 2008)<br>ib_mthca: Initializing 0000:00:00.0<br>PCI: Enabling
device 0000:00:00.0 (0000 -> 0002)<br>PCI: Setting latency
timer of device 0000:00:00.0 to 64<br>ib_mthca 0000:00:00.0:
QUERY_FW command failed, aborting.<br>ib_mthca: probe of
0000:00:00.0 failed with error -11<br><br>in dom0:<br>Feb 11
19:44:37 p128 kernel: tap tap-3-51712: 2 getting info<br>Feb 11
19:44:37 p128 kernel: pciback: vpci: 0000:0e:00.0: assign to
virtual slot 0<br>Feb 11 19:44:37 p128 kernel: device vif3.0
entered promiscuous mode<br>Feb 11 19:44:37 p128 kernel:
ADDRCONF(NETDEV_UP): vif3.0: link is not ready<br>Feb 11 19:44:39
p128 kernel: blktap: ring-ref 9, event-channel 9, protocol 1
(x86_64-abi)<br>Feb 11 19:44:48 p128 kernel: pciback 0000:0e:00.0:
Driver tried to write to a read-only configuration space field at
offset 0x44, size 2. This may be harmless, but if you have
problems with your device:<br>Feb 11 19:44:48 p128 kernel: 1) see
permissive attribute in sysfs<br>Feb 11 19:44:48 p128 kernel: 2)
report problems to the xen-devel mailing list along with details
of your device obtained from lspci.<br>Feb 11 19:44:48 p128
kernel: PCI: Enabling device 0000:0e:00.0 (0000 -> 0002)<br>Feb
11 19:44:48 p128 kernel: ACPI: PCI Interrupt 0000:0e:00.0[A] ->
GSI 16 (level, low) -> IRQ 16<br>Feb 11 19:44:49 p128 kernel:
ACPI: PCI interrupt for device 0000:0e:00.0
disabled<br><br><br><br>some more details - [root@p128 ~]# rpm -qa
| grep
xen<br>kernel-xen-2.6.18-92.1.22.el5<br>xen-3.0.3-64.el5_2.9<br>xen-libs-3.0.3-64.el5_2.9<br>xen-libs-3.0.3-64.el5_2.9<br><br>[root@p128
~]# ibv_devinfo<br>hca_id:
mthca0<br>
fw_ver:
5.3.0<br>
node_guid:
0002:c902:0022:cd48<br>
sys_image_guid:
0002:c902:0022:cd4b<br>
vendor_id:
0x02c9<br>
vendor_part_id:
25218<br>
hw_ver:
0x20<br>
board_id:
MT_0370130002<br>
phys_port_cnt:
2<br>
port:
1<br>
state:
PORT_INIT
(2)<br>
max_mtu:
2048
(4)<br>
active_mtu:
512
(2)<br>
sm_lid:
0<br>
port_lid:
0<br>
port_lmc:
0x00<br><br>
port:
2<br>
state:
PORT_DOWN
(1)<br>
max_mtu:
2048
(4)<br>
active_mtu:
512
(2)<br>
sm_lid:
0<br>
port_lid:
0<br>
port_lmc:
0x00<br><br><br>any help greatly
appreciated.<br><br>~subbu<br><br>
<div class="gmail_quote">On Sat, Oct 18, 2008 at 4:54 AM, David
Brown <span dir="ltr"><<a href="mailto:dmlb2000@gmail.com" target="_blank">dmlb2000@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Okay
so my question to the openfabrics guys is, why would the
OFED<br>drivers fail to read the firmware?<br><br>Any
thoughts?<br><br>Thanks,<br>- David Brown<br><br><br>----------
Forwarded message ----------<br>From: David Brown <<a href="mailto:dmlb2000@gmail.com" target="_blank">dmlb2000@gmail.com</a>><br>Date: Thu, Sep 11,
2008 at 2:24 PM<br>Subject: pciback module not working<br>To: <a href="mailto:xen-users@lists.xensource.com" target="_blank">xen-users@lists.xensource.com</a>, <a href="mailto:xen-devel@lists.xensource.com" target="_blank">xen-devel@lists.xensource.com</a><br>
<br><br>This
issue was brought up about a year and a half ago. So I'll
bring<br>it up again and see if anything happens.<br><br>I've
got an infiniband network and am attempting to pass
the<br>infiniband card through the host and give it to the
guest.<br>I'm working with standard CentOS 5.2 on both guest and
host with their<br>provided xen (3.0.3 ish). I've also attempted
to install the newest<br>Xen 3.3 and use their standard host
kernel and that did the same<br>thing. The guest dmesg output in
the guest is similar on both<br>permissive and normal
mode.<br><br>I'm getting issues with detecting the firmware on
the card for some reason...<br><br>Any help would be
appreciated.<br><br>Thanks,<br>- David Brown<br><br>=== GUEST
dmesg output ===<br>ib_mthca: Mellanox InfiniBand HCA driver
v1.0 (February 28, 2008)<br>ib_mthca: Initializing
0000:00:00.0<br>PCI: Enabling device 0000:00:00.0 (0000 ->
0002)<br>PCI: Setting latency timer of device 0000:00:00.0 to
64<br>ib_mthca 0000:00:00.0: QUERY_FW command failed,
aborting.<br>ib_mthca: probe of 0000:00:00.0 failed with error
-11<br>=======================<br><br>=== Host modprobe.conf
===<br>alias eth0 bnx2<br>alias eth1 bnx2<br>alias
scsi_hostadapter cciss<br>options pciback
hide=(41:00.0)<br>=====================<br><br>=== Host lspci
output ===<br># lspci -vs 41:00.0<br>41:00.0 InfiniBand:
Mellanox Technologies MT25204 [InfiniHost III Lx<br>HCA] (rev
20)<br> Subsystem: Hewlett-Packard Company
Unknown device 170a<br> Flags: fast devsel,
IRQ 16<br> Memory at fdc00000 (64-bit,
non-prefetchable) [disabled] [size=1M]<br>
Memory at fd000000 (64-bit, prefetchable) [disabled]
[size=8M]<br> Capabilities: [40] Power
Management version 2<br> Capabilities: [48]
Vital Product Data<br> Capabilities: [90]
Message Signalled Interrupts: 64bit+ Queue=0/5 Enable-<br>
Capabilities: [84] MSI-X: Enable- Mask-
TabSize=32<br> Capabilities: [60] Express
Endpoint IRQ 0<br>=====================<br><br>This makes sure
it get loaded first off before anything else.<br>=== Host
mkinitrd cmd ===<br># mkinitrd -f --with=pciback --preload
pciback<br>/boot/initrd-2.6.18-92.1.10.el5xen.img
2.6.18-92.1.10.el5xen<br>====================<br><br>=== Host
pciback dmesg ===<br>pciback 0000:41:00.0: Driver tried to write
to a read-only<br>configuration space field at offset 0x44, size
2. This may be<br>harmless, but if you have problems with your
device:<br>1) see permissive attribute in sysfs<br>2) report
problems to the xen-devel mailing list along with details
of<br>your device obtained from lspci.<br>PCI: Enabling device
0000:41:00.0 (0000 -> 0002)<br>ACPI: PCI Interrupt
0000:41:00.0[A] -> GSI 16 (level, low) -> IRQ 16<br>PCI:
Setting latency timer of device 0000:41:00.0 to 64<br>ACPI: PCI
interrupt for device 0000:41:00.0
disabled<br>======================<br><br>=== Host pciback dmesg
(after setting it permissive) ===<br>pciback 0000:41:00.0:
enabling permissive mode configuration space
accesses!<br>pciback 0000:41:00.0: permissive mode is
potentially unsafe!<br>pciback: vpci: 0000:41:00.0: assign to
virtual slot 0<br>device vif1.0 entered promiscuous
mode<br>ADDRCONF(NETDEV_UP): vif1.0: link is not
ready<br>blkback: ring-ref 9, event-channel 28, protocol 1
(x86_64-abi)<br>PCI: Enabling device 0000:41:00.0 (0000 ->
0002)<br>ACPI: PCI Interrupt 0000:41:00.0[A] -> GSI 16
(level, low) -> IRQ 16<br>PCI: Setting latency timer of
device 0000:41:00.0 to 64<br>ACPI: PCI interrupt for device
0000:41:00.0
disabled<br>=========================================<br><br>===
Guest lspci output ===<br># lspci -v<br>00:00.0 InfiniBand:
Mellanox Technologies MT25204 [InfiniHost III Lx<br>HCA] (rev
20)<br> Subsystem: Hewlett-Packard Company
Unknown device 170a<br> Flags: fast devsel,
IRQ 16<br> Memory at fdc00000 (64-bit,
non-prefetchable) [disabled] [size=1M]<br>
Memory at fd000000 (64-bit, prefetchable) [disabled]
[size=8M]<br> Capabilities: [40] Power
Management version 2<br> Capabilities: [48]
Vital Product Data<br> Capabilities: [90]
Message Signalled Interrupts: 64bit+<br>Queue=0/5
Enable-<br> Capabilities: [84] MSI-X:
Enable- Mask- TabSize=32<br> Capabilities:
[60] Express Endpoint IRQ
0<br>=====================<br>_______________________________________________<br>general
mailing list<br><a href="mailto:general@lists.openfabrics.org" target="_blank">general@lists.openfabrics.org</a><br><a href="http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general" target="_blank">http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general</a><br>
<br>To
unsubscribe, please visit <a href="http://openib.org/mailman/listinfo/openib-general" target="_blank">http://openib.org/mailman/listinfo/openib-general</a><br></blockquote></div><br><br clear="all"><br>-- <br>
. . . s u b b u<br>"You've got to be
original, because if you're like someone else, what do they need
you for?"<br></div></div></blockquote></div></blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original,
because if you're like someone else, what do they need you
for?"<br></div></div></blockquote></div></blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original,
because if you're like someone else, what do they need you
for?"<br></div></div></blockquote></div></blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original, because if
you're like someone else, what do they need you
for?"<br></div></div></blockquote></div>
</blockquote></div><br><br clear="all"><br>-- <br>. . . s u b b u<br>"You've got to be original, because if you're like someone else, what do they need you for?"<br>