[ofa-general] Multicast Performance

[Hal Rosenstock]

On Thu, 2008-05-29 at 15:35 +0200, Marcel Heinz wrote:
> Hal Rosenstock wrote:
> > On Thu, 2008-05-29 at 11:19 +0200, Marcel Heinz wrote:
> >>Dotan Barak wrote:
> >>>Marcel Heinz wrote:
> >>>
> >>>>[low multicast throughput of ~250MB/s with own benchmark tool]
> >>>
> >>>1) I know that ib_send_bw supports multicast as well, can you please
> >>>check that you can reproduce your problem
> >>>on this benchmark too?
> >>
> >>|  #bytes #iterations    BW peak[MB/sec]    BW average[MB/sec]
> >>|    2048        1000             301.12                247.05
> >>
> >>This is the same result as my own benchmark showed in that scenario.
> >>
> >>
> >>>2) You should expect that multicast messages will be slower than
> >>>unicast because the HCA/switch treat them in different way
> >>>(message duplication need to be done if needed).
> >>
> >>Yes, but 250MB/s vs. 1100MB/s (UD unicast throughput) seems to be a bit
> >>too much of overhead, don't you think?
> > 
> > 
> > Agreed.
> > 
> > 
> >>Especially if I take into account
> >>that with my own benchmark, I can get ~950MB/s when I start another
> >>receiver on the same host as the sender. Note that both of the
> >>receivers, the local and the remote one, are seeing all packets at that
> >>rate, so the HCAs and the switch must be able to handle multicast
> >>packets with this throughput.
> > 
> > 
> > Perhaps this is a static rate issue.
> > 
> > What SM is being used ?
> 
> It's OpenSM 3.1.7. I had also made some tests with OpensSM 3.2.1, but
> this didn't change anything.

Can you validate either the PathRecord or MCMemberRecord returned or the
static rate applied to the multicast QP in the various scenarios ? If it
is the same, this is not the problem but if it's different then we're on
to something here.

-- Hal

> Regards,
> Marcel
> 
> 
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
> 
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general