[ofiwg] DS/DA slide deck for discussion today

Atchley, Scott atchleyes at ornl.gov
Tue Dec 22 05:14:47 PST 2015


Hi Paul,

I do not intend to call in today. I would suggest the following to replace the rationale:

* Block and object storage protocols do not map well to sockets
  - Reliable sockets are stream-oriented and require markers.
  - Datagram sockets map well to block access at the expense of reliability.
  - Sockets require implicit buffering at the sender and receiver, which adds latency, increases CPU utilization, and reduces throughput.
  - Sockets’ send completion semantics are limited. Socket sends only guarantee that the data is buffered locally.
  - Sockets do not provide a mechanism for one-sided access (i.e. requires an active process on both the sender and receiver).
  - Stream socket connections do not allow multiple thread access without external synchronization (i.e. a mutex to ensure single thread access to partial messages and to avoid losing markers).

* Block and object storage protocols map well to reliable, message-based APIs that also provide RMA access.
  - kfabric provides reliable and unreliable messages; the process do not need to maintain message markers.
  - kfabric does not require implicit buffering.
  - kfabric provides a richer set of send completion semantics (e.g. local completion, remote completion, others?).
  - kfabric “connection” are thread-safe; multiple threads can progress them independently.
  - kfabric provides one-sided semantics that can allow direct hardware access without CPU intervention.


> On Dec 22, 2015, at 6:34 AM, Paul Grun <grun at cray.com> wrote:
> 
>  
>  
> Cray Inc.
> Office:    (503) 620-8757
> Mobile:  (503) 703-5382
>  
> <kfabric-maintainer discussion__2015_1222.pptx>_______________________________________________
> ofiwg mailing list
> ofiwg at lists.openfabrics.org
> http://lists.openfabrics.org/mailman/listinfo/ofiwg



More information about the ofiwg mailing list