[openib-general] vstat error on bproc slave node (VAPI_EGEN)

[gshipman]

I am relatively new to openib so here goes:

I am attempting to configure our small cluster to use bproc and openib. 
Note I am using gen1 on kernel 2.6.6 patched with the clustermatic 
stuff, (should I be using gen2, is it stable for general use?).

I have successfully gotten things going on the head node including 
opensm. I have successfully gotten the slave nodes to run the patched 
kernel, load the appropriate modules as well as the various user level 
libraries but I am having an issue on the slave nodes:

If I run:
$bpsh 13 /usr/mellanox/bin/vstat
1 HCA found:
         hca_id=InfiniHost0
Error: Could not retrieve handle to the HCA InfiniHost0 (VAPI_EGEN)


On the head node I get:
$/usr/mellanox/bin/vstat
1 HCA found:
         hca_id=InfiniHost0
         vendor_id=0x02C9
         vendor_part_id=0x5A44
         hw_ver=0xA1
         fw_ver=0x300020000
         num_phys_ports=2
                 port=1
                 port_state=PORT_DOWN
                 sm_lid=0x0000
                 port_lid=0x0353
                 port_lmc=0x00
                 max_mtu=2048

                 port=2
                 port_state=PORT_ACTIVE
                 sm_lid=0x0354
                 port_lid=0x0354
                 port_lmc=0x00
                 max_mtu=2048

I can run ifconfig on the slave I see ib0 properly:
$bpsh 13 ifconfig ib0
ib0       Link encap:Ethernet  HWaddr 00:00:00:00:00:00
           BROADCAST MULTICAST  MTU:2044  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:128
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Thanks,

Galen