[ofa-general] Multicast Performance

[Marcel Heinz]

Hi Olga,

Olga Shern (Voltaire) wrote:
> Hi Marcel,
>
> We also did tests with multicast traffic and figured out that when
> using Arbel HCAs

As far as I can tell, we're also using "Arbel" HCAs.

> there is performance penalty when:
> 1. Sending MC packets with NO QP attached on local Arble HCA.
> 2. Receiving MC packet with more then 1 QP attached on a single HCA
> which causes the back pressure that slows down the sender.
>
> Hope this helps

This explains some of the effects I've seen. We only need one sender and
n receivers, all on separate hosts. I've only tested all that other
scenarios because I wanted to find out more about why the performance
was so bad in that original scenario. Attaching the send QP to the group
as a workaround shouldn't be a problem. However, my numbers suggest that
attaching alone isn't enough, the data has to be received actually,
which hasn't any negative impact on my benchmark, but is consuming up to
another 1GByte/s of memory bandwidth, which could hurt the real application.

Besides of that, there are still performance problems with n > 1, as I
mentioned in the original post (c is the client sending the data, s is
the server, receiving):

| We could test on another cluster with 6 nodes (also with MT25204 HCAs,
| I don't know the OFED version and switch type) and got the following
| results:
|
| Host1	Host2	Host3	Host4	Host5	Host6	Throughput (MByte/s)	
| 1s	 1s 				 1c	 255,15
| 1s     1s      1s 			 1c 	 255,22
| 1s	 1s	 1s	 1s		 1c 	 255,22
| 1s	 1s 	 1s	 1s	 1s	 1c	 255,22
|
| 1s1c	 1s	 1s				 738,64
| 1s1c	 1s 	 1s	 1s			 695,08
| 1s1c	 1s 	 1s	 1s 	 1s		 565,14
| 1s1c	 1s 	 1s 	 1s	 1s     1s	 451,90

But maybe that was a problem of the switch or general setup. I will make
some tests with our own switch as soon as I got installed an additional
host. Testing multicast with only two hosts is a bit odd... ;-)

Thank you!

Regards,
Marcel