[ofa-general] Multicast Performance

[Marcel Heinz]

Hal Rosenstock wrote:
> On Thu, 2008-05-29 at 11:19 +0200, Marcel Heinz wrote:
>>Dotan Barak wrote:
>>>Marcel Heinz wrote:
>>>
>>>>[low multicast throughput of ~250MB/s with own benchmark tool]
>>>
>>>1) I know that ib_send_bw supports multicast as well, can you please
>>>check that you can reproduce your problem
>>>on this benchmark too?
>>
>>|  #bytes #iterations    BW peak[MB/sec]    BW average[MB/sec]
>>|    2048        1000             301.12                247.05
>>
>>This is the same result as my own benchmark showed in that scenario.
>>
>>
>>>2) You should expect that multicast messages will be slower than
>>>unicast because the HCA/switch treat them in different way
>>>(message duplication need to be done if needed).
>>
>>Yes, but 250MB/s vs. 1100MB/s (UD unicast throughput) seems to be a bit
>>too much of overhead, don't you think?
> 
> 
> Agreed.
> 
> 
>>Especially if I take into account
>>that with my own benchmark, I can get ~950MB/s when I start another
>>receiver on the same host as the sender. Note that both of the
>>receivers, the local and the remote one, are seeing all packets at that
>>rate, so the HCAs and the switch must be able to handle multicast
>>packets with this throughput.
> 
> 
> Perhaps this is a static rate issue.
> 
> What SM is being used ?

It's OpenSM 3.1.7. I had also made some tests with OpensSM 3.2.1, but
this didn't change anything.

Regards,
Marcel