[openib-general] ip over ib throughtput

[Andras.Horvath at cern.ch]

Hello,

> My guess is ia64 MSI support is somewhat broken. :(

I'm using 'modprobe ib_mthca msi_x=1', and 'MSI INIT SUCCESS' is there
but /proc/interrupts doesn't show MSI being used so I'm puzzled a bit.
See dmesg output below.

However: 2.6.10-mm2, hp rx2600 (ia64), PCI-X HCA (mthca), iperf one-way
transfers:

		senderCPU	recvCPU		bandwidth
1 thread	18%		30%		2.03 Gbits/sec
5 threads	55%		78%		2.74 Gbits/sec
10 threads	50%		74%		2.71 Gbits/sec

(by the way, it was a nice xmas gift to see IB drivers in the 2.6.10-mm*
patches :) thanks!)

So, could someone clear up the MSI confusion please?

thanks,
Andras

in dmesg:

ib_mthca 0000:81:00.0: Found bridge: Mellanox Technology MT23108 PCI Bridge (0000:80:01.0)
MSI INIT SUCCESS
ib_mthca 0000:81:00.0: FW version 000100180000, max commands 1
ib_mthca 0000:81:00.0: FW size 6143 KB (start cfa00000, end cfffffff)
ib_mthca 0000:81:00.0: HCA memory size 131071 KB (start c8000000, end cfffffff)
ib_mthca 0000:81:00.0: Max QPs: 16777216, reserved QPs: 16, entry size: 256
ib_mthca 0000:81:00.0: Max CQs: 16777216, reserved CQs: 128, entry size: 64
ib_mthca 0000:81:00.0: Max EQs: 64, reserved EQs: 1, entry size: 64
ib_mthca 0000:81:00.0: reserved MPTs: 16, reserved MTTs: 16
ib_mthca 0000:81:00.0: Max PDs: 16777216, reserved PDs: 0, reserved UARs: 1
ib_mthca 0000:81:00.0: Max QP/MCG: 16777216, reserved MGMs: 0
ib_mthca 0000:81:00.0: Flags: 003f0337
ib_mthca 0000:81:00.0: profile[ 0]--10/20 @ 0x        c8000000 (size 0x 4000000)
ib_mthca 0000:81:00.0: profile[ 1]-- 0/16 @ 0x        cc000000 (size 0x 1000000)
ib_mthca 0000:81:00.0: profile[ 2]-- 7/18 @ 0x        cd000000 (size 0x  800000)
ib_mthca 0000:81:00.0: profile[ 3]-- 9/17 @ 0x        cd800000 (size 0x  800000)
ib_mthca 0000:81:00.0: profile[ 4]-- 3/16 @ 0x        ce000000 (size 0x  400000)
ib_mthca 0000:81:00.0: profile[ 5]-- 4/16 @ 0x        ce400000 (size 0x  200000)
ib_mthca 0000:81:00.0: profile[ 6]--12/15 @ 0x        ce600000 (size 0x  100000)
ib_mthca 0000:81:00.0: profile[ 7]-- 8/13 @ 0x        ce700000 (size 0x   80000)
ib_mthca 0000:81:00.0: profile[ 8]--11/ 9 @ 0x        ce780000 (size 0x    4000)
ib_mthca 0000:81:00.0: profile[ 9]-- 6/ 5 @ 0x        ce784000 (size 0x     800)
ib_mthca 0000:81:00.0: HCA memory: allocated 106002 KB/124928 KB (18926 KB free)
ib_mthca 0000:81:00.0: Allocated EQ 1 with 65536 entries
ib_mthca 0000:81:00.0: Allocated EQ 2 with 128 entries
ib_mthca 0000:81:00.0: Allocated EQ 3 with 128 entries
ib_mthca 0000:81:00.0: Setting mask 00000000000343fe for eqn 2
ib_mthca 0000:81:00.0: Setting mask 0000000000000400 for eqn 3
ib_mthca 0000:81:00.0: Port change to active for port 1

in /proc/interrupts:

 57:   38401561          0  IO-SAPIC-level  ib_mthca