[ofa-general] MT25204 versus MT25208

[Bart Van Assche]

Hello,

Four hosts are connected through an InfiniBand network, two with a
MT25204 HCA and two with a MT25208 HCA. All four hosts communicate
fine via IPoIB, and iperf shows the expected results. Until now I only
succeeded to run ib_rdma_bw between the MT25204 interfaces. When I run
ib_rdma_bw on one of the MT25208 interfaces a strange error message is
printed. Any idea how I can fix this ?

MT25204 to MT25204:
root at 192.168.102.5:~# ib_rdma_bw
6467: | port=18515 | ib_port=1 | size=65536 | tx_depth=100 |
iters=1000 | duplex=0 | cma=0 |
6467: Local address:  LID 0x06, QPN 0x140405, PSN 0x355f4d RKey
0x22002400 VAddr 0x002ad73ba76000
6467: Remote address: LID 0x11, QPN 0x80405, PSN 0xa189f4, RKey
0x40002400 VAddr 0x002aff033e3000
root at 192.168.102.12:~# ib_rdma_bw 192.168.102.5
6857: | port=18515 | ib_port=1 | size=65536 | tx_depth=100 |
iters=1000 | duplex=0 | cma=0 |
6857: Local address:  LID 0x11, QPN 0x80405, PSN 0xa189f4 RKey
0x40002400 VAddr 0x002aff033e3000
6857: Remote address: LID 0x06, QPN 0x140405, PSN 0x355f4d, RKey
0x22002400 VAddr 0x002ad73ba76000
6857: Bandwidth peak (#0 to #989): 674.73 MB/sec
6857: Bandwidth average: 674.711 MB/sec
6857: Service Demand peak (#0 to #989): 2887 cycles/KB
6857: Service Demand Avg  : 2887 cycles/KB

MT25208 to MT25204:
root at 192.168.102.5:~# ib_rdma_bw
6468: | port=18515 | ib_port=1 | size=65536 | tx_depth=100 |
iters=1000 | duplex=0 | cma=0 |
server read: Success
6468:pp_server_exch_dest: 0/45 Couldn't read remote address
root at 192.168.102.100:~# ib_rdma_bw 192.168.102.5
6466: | port=18515 | ib_port=1 | size=65536 | tx_depth=100 |
iters=1000 | duplex=0 | cma=0 |
6466:main: Local lid 0x0 detected. Is an SM running?

Software versions:
* Linux kernel 2.6.22.9, 64-bit version.
* OFED-1.2.5.4 userspace components.

Bart.