[openib-general] [ANNOUNCE] ContributeRDS(ReliableDatagramSockets) to OpenIB

[Ranjit Pandit]

> I agree that there is some improvement in going from one QP per
> UDP socket to one per node, but it still will likely not
> scale to 10,000 node clusters, which is something that Oracle
> probably does not care about, but others in HPC do.
>

To put the improvement in perspective:

For Mpi running on a 10,000 node cluster with 2 or 4 way nodes, here
are the QP/ CM connection requirements: (assuming intra node
communication doesn't use IB)

Procs per node   uDapl/Sdp      Rds
2                       19996            9999
4                       39984            9999

Clearly, there is tradeoff in performance as we go from uDapl/Sdp to
Rds. The choice will have to depend on the requirements of performance
Vs Scalability.
Btw, for this large a cluster, there is a huge overhead in just
setting up the connections. Rds connections are setup only once.

> If we are going to invent a Reliable Datagram Service, shouldn't
> it be made to scale so that MPIs that currently use datagrams
> could also benefit ?
>
> woody
>
>
>
>
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>