[ofiwg] flow control in the reliable-datagram provider

[Hefty, Sean]

This is a follow up email from today's ofiwg.

We are gathering input on the best mechanisms for implementing flow control as part of a reliable datagram provider that sends and receives unreliable packets.  The mechanism must be able to work at large scales (thousands to tens of thousands of nodes).  Hopefully some of the MPI folks have done work in this area and have an already awesome solution.  :)

Based on searches, these are some of the ideas being looked at:

* Starting with sliding window protocol with selective repeat.
* Credit based scheme per message.
* Credit based scheme per peer.
* Combined per message and per peer credits.
* Selective ACKs - likely based off bit vectors.
* Separating flows across multiple datagram endpoints.
* Pause and/or choke feedback packets.
* Data rate limits (e.g. leaky bucket algorithm)
* Optional request for ACK response

Any input, particularly when dealing at scale or many-to-one communication patters, would be helpful! Obviously, bonus points for implementation simplicity and low-overhead.

- Sean