[openib-general] [ANNOUNCE] ContributeRDS(ReliableDatagramSockets) to OpenIB

[Caitlin Bestler]

> -----Original Message-----
> From: openib-general-bounces at openib.org 
> [mailto:openib-general-bounces at openib.org] On Behalf Of Bob Woodruff
> Sent: Friday, November 04, 2005 10:58 AM
> To: 'Fab Tillier'; 'Sean Hefty'; Rick Frank
> Cc: openib-general at openib.org
> Subject: RE: [openib-general] [ANNOUNCE] 
> ContributeRDS(ReliableDatagramSockets) to OpenIB
> 
> Fab wrote,
> >There is not a 1:1 relationship between a UDP application 
> socket and an 
> >IB
> QP,
> >rather there is a single IB connection between systems over which 
> >traffic
> from
> >multiple UDP sockets flows.
> 
> >- Fab
> 
> That would probably provide better scalability, since there 
> would not be a 1:1 mapping between UDP sockets and IB 
> connections, however for large clusters there may still be a 
> scalability issue if every node needs to have a connection to 
> every other node. 
> If you implemented it on top of datagrams instead, then each 
> node would only need one QP, rather than one for every node 
> in the cluster.
> 

But then the application would have to take responsibility for
congestion control and retries after network packet losses.

RDS allows an application all the benefits of a reliable connection
without the overhead, except for per connection back-pressure.
Many applications do not need pre-connection back-pressure since
they already have session-wide flow control policies in place.

Going from one connection for each pair of application endpoints
to one connection for each pair of hosts is a major improvement.
For most applications going down to a single QP after that is
not sufficiently valuable to add the complexity of working over
a totally unreliable protocol.