[ofa-general] Re: msi-x seems to cause large performance variation for connectx / datagram mode

[Or Gerlitz]

> digging a little more, I see now that if I disable msi-x, the
> performance gets to a fixed value of about 620MB/s with connectx
> which is the lower range that msi-x gives me...

setting the irq daemon off, I see that if I set the cpu affinity of the netperf
process to be as the one of the mlx4 irq I get fixed performance of 650 MB/s

If I set it to different CPU, then eventually I get 480MB/s constantly and
620MB/s constantly if is a different core on the same CPU...

HCA			MB/s	msi_x	irqdaemon	netperf affinity
=======================================================================
ConnectX-ConnectX	620     0 	on		none

ConnectX-ConnectX	620-850 1 	on		none
ConnectX-ConnectX	650     1 	off		same as irq
ConnectX-ConnectX	620     1 	off		different core
ConnectX-ConnectX	480     1 	off		different cpu


Or.