[openib-general] could not add HCA InfiniHost0

QiWang, Chen QiWang.Chen at Clustars.CN
Thu Sep 15 03:11:26 PDT 2005


Hello everyone,
I had Mellanox MT23108 HCA, RHEL4 U1, kernel 2.6.9-11,
node1 to node8 works fine. Drivers: IBGD-1.8.0, FW= 3.3.3
lspci:
02:00.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge(reva1)
03:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)

but node9 to node16 doesn't work.
lspci:
02:01.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge(reva1)
03:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)

there are some diff:
02:00.0 --> work
02:01.0 --> failed

and first time I install the ib-verbs on node1, It also failed, because 
lspci= 02:01.0, an I don not know how i change 02:01.0 to 02:00.0, and 
it works fine for me.

the error logs list here:
_---------------------------------------------------------------------

Hostname: c01-14
OS: Red Hat Enterprise Linux AS release 4 (Nahant Update 1)
Kernel \r on an \m
Current kernel: 2.6.9-11.ELsmp
Architecture: i686      
GCC version: gcc (GCC) 3.4.3 20050227 (Red Hat 3.4.3-22.1)
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is
NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.
CPU: model name : Intel(R) Xeon(TM) CPU 2.66GHz
MemTotal:      2075004 kB 
Chipset: 00.0 Host bridge: Intel Corporation E7501 Memory Controller Hub
(rev 01)


Device /dev/mst/mt23108_pci_cr0 Info:
Firmware:
        Version:        3.03.0003
        Date:   05/07/2005 18:46:35


############# LSPCI ##############
00:00.0 Host bridge: Intel Corporation E7501 Memory Controller Hub (rev
01)
00:00.1 Class ff00: Intel Corporation E7500/E7501 Host RASUM Controller
(rev 01)
00:04.0 PCI bridge: Intel Corporation E7500/E7501 Hub Interface D PCI-
to-PCI Bridge (rev 01)
00:1d.0 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #1) (rev
02)
00:1d.1 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #2) (rev
02)
00:1d.2 USB Controller: Intel Corporation 82801CA/CAM USB (Hub #3) (rev
02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 42)
00:1f.0 ISA bridge: Intel Corporation 82801CA LPC Interface Controller
(rev 02)
00:1f.1 IDE interface: Intel Corporation 82801CA Ultra ATA Storage
Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801CA/CAM SMBus Controller (rev 02)
01:1c.0 PIC: Intel Corporation 82870P2 P64H2 I/OxAPIC (rev 04)
01:1d.0 PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge (rev
04)
01:1e.0 PIC: Intel Corporation 82870P2 P64H2 I/OxAPIC (rev 04)
01:1f.0 PCI bridge: Intel Corporation 82870P2 P64H2 Hub PCI Bridge (rev
04)
02:01.0 PCI bridge: Mellanox Technologies MT23108 PCI Bridge (rev a1)
03:00.0 InfiniBand: Mellanox Technologies MT23108 InfiniHost (rev a1)
04:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S
Gigabit Ethernet (rev 03)
04:01.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704S
Gigabit Ethernet (rev 03)
05:00.0 VGA compatible controller: Chips and Technologies F69030 (rev
61)
05:08.0 Ethernet controller: Intel Corporation 82801BA/BAM/CA/CAM
Ethernet Controller (rev 42)

############# LSPCI -N ##############
00:00.0 Class 0600: 8086:254c (rev 01)
00:00.1 Class ff00: 8086:2541 (rev 01)
00:04.0 Class 0604: 8086:2547 (rev 01)
00:1d.0 Class 0c03: 8086:2482 (rev 02)
00:1d.1 Class 0c03: 8086:2484 (rev 02)
00:1d.2 Class 0c03: 8086:2487 (rev 02)
00:1e.0 Class 0604: 8086:244e (rev 42)
00:1f.0 Class 0601: 8086:2480 (rev 02)
00:1f.1 Class 0101: 8086:248b (rev 02)
00:1f.3 Class 0c05: 8086:2483 (rev 02)
01:1c.0 Class 0800: 8086:1461 (rev 04)
01:1d.0 Class 0604: 8086:1460 (rev 04)
01:1e.0 Class 0800: 8086:1461 (rev 04)
01:1f.0 Class 0604: 8086:1460 (rev 04)
02:01.0 Class 0604: 15b3:5a46 (rev a1)
03:00.0 Class 0c06: 15b3:5a44 (rev a1)
04:01.0 Class 0200: 14e4:16a8 (rev 03)
04:01.1 Class 0200: 14e4:16a8 (rev 03)
05:00.0 Class 0300: 102c:0c30 (rev 61)
05:08.0 Class 0200: 8086:2449 (rev 42)


############# LSMOD ##############
Module                  Size  Used by
ib_sa_client           34312  0
ib_client_query        22240  1 ib_sa_client
ib_poll                21560  1 ib_client_query
ib_useraccess          16708  0
ib_tavor               39972  0
ib_mad                 26380  3 ib_client_query,ib_useraccess,ib_tavor
ib_core               237588  4
ib_sa_client,ib_useraccess,ib_tavor,ib_mad
ib_services            22468  7
ib_sa_client,ib_client_query,ib_poll,ib_useraccess,ib_tavor,ib_mad,ib_core
mod_thh               290020  0
mst_pciconf            87296  0
mst_pci                84352  0
mod_vip               329288  2 ib_tavor,mod_thh
mlxsys                 95664  2 mod_thh,mod_vip
nfs                   200869  0
nfsd                  205281  9
exportfs               10049  1 nfsd
lockd                  65257  3 nfs,nfsd
md5                     8001  1
ipv6                  238817  12
autofs4                22085  2
sunrpc                138789  20 nfs,nfsd,lockd
dm_mod                 58949  0
button                 10449  0
battery                12869  0
ac                      8773  0
uhci_hcd               32729  0
tg3                    82373  0
e100                   36673  0
mii                     8641  1 e100
floppy                 58065  0
ext3                  118729  3
jbd                    59481  1 ext3

############# DMESG ##############
iband/ib_verbs/hw/mellanox-hca/mlxhh/thh/cmdif_comm.c[1482]:
print_track_arr: idx=31, token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=32, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=33, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=34, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=35, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=36, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=37, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=38, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=39, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=40, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=41, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=42, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=43, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=44, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=45, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=46, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=47, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=48, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=49, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=50, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=51, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=52, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=53, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=54, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=55, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=56, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=57, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=58, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=59, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=60, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=61, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=62, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=63, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=64, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=65, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=66, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=67, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=68, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=69, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=70, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=71, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=72, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=73, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=74, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=75, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=76, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=77, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=78, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=79, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=80, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=81, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=82, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=83, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=84, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=85, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=86, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=87, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=88, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=89, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=90, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=91, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=92, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=93, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=94, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=95, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=96, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=97, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=98, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=99, token=0x0000,
counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=100,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=101,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=102,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=103,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=104,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=105,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=106,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=107,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=108,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=109,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=110,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=111,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=112,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=113,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=114,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=115,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=116,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=117,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=118,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=119,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=120,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=121,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=122,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=123,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=124,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=125,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=126,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=127,
token=0x0000, counter=0
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif.c[211]: Failed command 0x24 (TAVOR_IF_CMD_MAD_IFC):
status=0x103 (0x0103 - unexpected error - fatal)

 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[250]: XHH_hob_query_port_prop: cmdif returned
FATAL
 VIPKL(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/vip/qpm.c[291]: QPM_new: HOBKL_query_port_prop returned with error:
-254 = VAPI_EFATAL
 VIPKL(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/vip/qpm.c[322]: QPM_new:  returned with error: -254 = VAPI_EFATAL
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[2323]: XHH_hob_halt_hca: HALT HCA returned
0x103
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[2699]: XHH_hob_restart: destroying old HOB
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob.c[1581]: XHH_hob_destroy_internal: FATAL ERROR
 THH(1): var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[2705]: XHH_hob_restart: creating new HOB

Mellanox Tavor Device Driver is creating device "InfiniHost0" (bus=03,
devfn=00)

[KERNEL_IB][_tsIbTavorInitOne][/var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/provider/tavor_main.c:178]InfiniHost0: VAPI_open_hca failed, status -254 (Fatal error (Local Catastrophic Error))
[KERNEL_IB][_tslbTavorPnPEventHandler][/var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/provider/tavor_main.c:352]_tslbTavorPnPEventHandler: could not add HCA InfiniHost0 (-19)

############# Messages ##############
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=106,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=107,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=108,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=109,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=110,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=111,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=112,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=113,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=114,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=115,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=116,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=117,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=118,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=119,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=120,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=121,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=122,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=123,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=124,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=125,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=126,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif_comm.c[1482]: print_track_arr: idx=127,
token=0x0000, counter=0
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/cmdif.c[211]: Failed command 0x24 (TAVOR_IF_CMD_MAD_IFC):
status=0x103 (0x0103 - unexpected error - fatal)
Sep 15 01:08:54 c01-14 kernel:
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[250]: XHH_hob_query_port_prop: cmdif returned
FATAL
Sep 15 01:08:54 c01-14 kernel:  VIPKL(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-hca/vip/qpm.c
[291]: QPM_new: HOBKL_query_port_prop returned with error: -254 =
VAPI_EFATAL
Sep 15 01:08:54 c01-14 kernel:  VIPKL(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-hca/vip/qpm.c
[322]: QPM_new:  returned with error: -254 = VAPI_EFATAL
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[2323]: XHH_hob_halt_hca: HALT HCA returned
0x103
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[2699]: XHH_hob_restart: destroying old HOB
Sep 15 01:08:54 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob.c[1581]: XHH_hob_destroy_internal: FATAL ERROR
Sep 15 01:08:55 c01-14 kernel:  THH(1):
var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/mellanox-
hca/mlxhh/thh/hob_comm.c[2705]: XHH_hob_restart: creating new HOB
Sep 15 01:08:55 c01-14 kernel:
Sep 15 01:08:55 c01-14 kernel: Mellanox Tavor Device Driver is creating
device "InfiniHost0" (bus=03, devfn=00)
Sep 15 01:08:55 c01-14 kernel:
Sep 15 01:08:56 c01-14 kernel:
[KERNEL_IB][_tsIbTavorInitOne][/var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/provider/tavor_main.c:178]InfiniHost0: VAPI_open_hca failed, status -254 (Fatal error (Local Catastrophic Error))
Sep 15 01:08:56 c01-14 kernel:
[KERNEL_IB][_tslbTavorPnPEventHandler][/var/tmp/IBGD//tmp/openib/infiniband/ib_verbs/hw/provider/tavor_main.c:352]_tslbTavorPnPEventHandler: could not add HCA InfiniHost0 (-19)
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error inserting ib_ipoib
(/lib/modules/2.6.9-11.ELsmp/kernel/drivers/infiniband/ib_ipoib.ko): No
such device
Sep 15 01:08:56 c01-14 modprobe: FATAL: Error running install command
for ib_ipoib

############# Running Processes ##############
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 00:32 ?        00:00:00 init [3]
root         2     1  0 00:32 ?        00:00:00 [migration/0]
root         3     1  0 00:32 ?        00:00:00 [ksoftirqd/0]
root         4     1  0 00:32 ?        00:00:00 [migration/1]
root         5     1  0 00:32 ?        00:00:00 [ksoftirqd/1]
root         6     1  0 00:32 ?        00:00:00 [events/0]
root         7     1  0 00:32 ?        00:00:00 [events/1]
root         8     6  0 00:32 ?        00:00:00 [khelper]
root         9     6  0 00:32 ?        00:00:00 [kacpid]
root        36     6  0 00:32 ?        00:00:00 [kblockd/0]
root        37     6  0 00:32 ?        00:00:00 [kblockd/1]
root        47     6  0 00:32 ?        00:00:00 [pdflush]
root        50     6  0 00:32 ?        00:00:00 [aio/0]
root        51     6  0 00:32 ?        00:00:00 [aio/1]
root        38     1  0 00:32 ?        00:00:00 [khubd]
root        49     1  0 00:32 ?        00:00:00 [kswapd0]
root       124     1  0 00:32 ?        00:00:00 [kseriod]
root       198     1  0 00:32 ?        00:00:00 [kjournald]
root      1020     1  0 00:32 ?        00:00:00 udevd
root      1331     1  0 00:32 ?        00:00:00 [kjournald]
root      1332     1  0 00:32 ?        00:00:00 [kjournald]
root      1723     1  0 00:32 ?        00:00:00 /sbin/dhclient -1 -q -
lf /var/lib/dhcp/dhclient-eth0.leases -pf /var/run/dhclient-eth0.pid
eth0
root      1782     1  0 00:32 ?        00:00:00 syslogd -m 0
root      1787     1  0 00:32 ?        00:00:00 klogd -x
root      1798     1  0 00:32 ?        00:00:00 irqbalance
rpc       1816     1  0 00:32 ?        00:00:00 portmap
rpcuser   1836     1  0 00:32 ?        00:00:00 rpc.statd
root      1931     1  0 00:32 ?        00:00:00 rpc.idmapd
root      1987     1  0 00:32 ?        00:00:00 ypbind
root      2137     1  0 00:32 ?        00:00:00 /usr/sbin/automount --
timeout=60 /home yp auto.home
root      2139     1  0 00:32 ?        00:00:00 /usr/sbin/automount --
timeout=60 /export yp auto.export
root      2157     1  0 00:32 ?        00:00:00 /usr/sbin/smartd
root      2167     1  0 00:32 ?        00:00:00 /usr/sbin/acpid
root      2267     1  0 00:32 ?        00:00:00 /usr/sbin/sshd
root      2282     1  0 00:32 ?        00:00:00 xinetd -stayalive -
pidfile /var/run/xinetd.pid
ntp       2298     1  0 00:32 ?        00:00:00 ntpd -u ntp:ntp -
p /var/run/ntpd.pid
root      2312     1  0 00:32 ?        00:00:00 rpc.rquotad
root      2321     1  0 00:32 ?        00:00:00 [nfsd]
root      2322     1  0 00:32 ?        00:00:00 [nfsd]
root      2323     1  0 00:32 ?        00:00:00 [nfsd]
root      2324     1  0 00:32 ?        00:00:00 [nfsd]
root      2325     1  0 00:32 ?        00:00:00 [nfsd]
root      2326     1  0 00:32 ?        00:00:00 [nfsd]
root      2327     1  0 00:32 ?        00:00:00 [nfsd]
root      2328     1  0 00:32 ?        00:00:00 [nfsd]
root      2329     1  0 00:32 ?        00:00:00 [lockd]
root      2330     1  0 00:32 ?        00:00:00 [rpciod]
root      2334     1  0 00:32 ?        00:00:00 rpc.mountd
nobody    2362     1  0 00:32 ?        00:00:00 /usr/sbin/gmond
root      2378     1  0 00:32 ?        00:00:00 gpm -m /dev/input/mice -
t imps2
root      2425     1  0 00:32 ?        00:00:00 /sbin/dhclient -1 -q -
lf /var/lib/dhcp/dhclient-eth0.leases -pf /var/run/dhclient-eth0.pid
eth0
root      2450     1  0 00:32 ?
00:00:00 /opt/torque-1.2.0p5/sbin/pbs_mom -r
root      2459     1  0 00:32 ?        00:00:00 crond
xfs       2487     1  0 00:32 ?        00:00:00 xfs -droppriv -daemon
root      2497     1  0 00:32 ?        00:00:00 anacron -s
daemon    2506     1  0 00:32 ?        00:00:00 /usr/sbin/atd
dbus      2516     1  0 00:32 ?        00:00:00 dbus-daemon-1 --system
root      2527     1  0 00:32 ?        00:00:00 cups-config-daemon
root      2538     1  0 00:32 ?        00:00:00 hald
root      2547     1  0 00:32 tty1     00:00:00 /sbin/mingetty tty1
root      2548     1  0 00:32 tty2     00:00:00 /sbin/mingetty tty2
root      2549     1  0 00:32 tty3     00:00:00 /sbin/mingetty tty3
root      2550     1  0 00:32 tty4     00:00:00 /sbin/mingetty tty4
root      2551     1  0 00:32 tty5     00:00:00 /sbin/mingetty tty5
root      2552     1  0 00:32 tty6     00:00:00 /sbin/mingetty tty6
root      3687     6  0 01:05 ?        00:00:00 [pdflush]
root      3855  2282  0 01:07 ?        00:00:00 in.rshd
root      3856  3855  0 01:07 ?
00:00:00 /bin/bash /etc/rc.d/init.d/openibd start
root      4109     1  0 01:08 ?        00:00:00 [cleanup_thread]
root      4136     1  0 01:08 ?        00:00:00 [ts_poll]
root      4235  3856  0 01:08 ?        00:00:00 /bin/ps -ef
##############################################

Can anybody help me???
Thx





-- 
QiWang, Chen <QiWang.Chen at Clustars.CN>
Clustars Supercomputing Technology corp.
http://www.Clustars.CN
TEL:+86-0816-2546345-815
FAX:+86-0816-2546370
Mobile:+86-13096497499




More information about the general mailing list