[ofiwg] DS/DA discussion

[Paul Grun]

I don't think we yet have a clear understanding of the requirements for user mode accesses to NVM.  This is on what DS/DA is currently focused.  Once we do, we will be in a better position to figure out how (or whether) to augment libfabric as needed.  Once we get to that point, I expect a lively exchange between DS/DA and OFI WG.

I think Scott's characterization of NVM-based storage as a sort of passive appliance, and contrasting that with process-to-process communication, is helpful because it gives us a model to work with that may not have been at front of mind for OFI WG when we first began discussing libfabric.  I think it is very helpful to have a group that is focused on APIs that meet the needs of storage and I argue that persistent memory is a form of storage.

The important things for now are twofold:  1) as Sean points out, the key is to ensure that APIs are developed using the principles of application-centric design which is how the work of OFI writ large has been conducted from the very beginning.  2) let's agree to leave discussion of data access to NVM in the DS/DA group, with the certainty that it will eventually come back to joint discussion between the two groups.

-Paul

-----Original Message-----
From: ofiwg-bounces at lists.openfabrics.org [mailto:ofiwg-bounces at lists.openfabrics.org] On Behalf Of Atchley, Scott
Sent: Tuesday, October 06, 2015 11:31 AM
To: Hefty, Sean
Cc: ofiwg at lists.openfabrics.org
Subject: Re: [ofiwg] DS/DA discussion

On Oct 6, 2015, at 2:24 PM, Hefty, Sean <sean.hefty at intel.com> wrote:

>> The original interface assumes process-to-process communication. I am 
>> simply wondering if that was too narrow for the storage aspect. Can 
>> the current interface handle completely passive resources? There is 
>> no need to "commit" memory in the process-to-process model, but the 
>> storage model might.
> 
> I somewhat disagree here.  As you mentioned, RMA and atomics interfaces do not assume process-to-process communication.

We agree, which is why I said that they are appropriate for storage. Message queues and tag matching are not.

> I'm also not sure about the definition of completely passive.  A CM 
> protocol requires some sort of message exchange, and reliability 
> requires ACKs.  Libfabric does assume network communication, so 
> probably isn't appropriate for local storage.  (I'm a fan of local 
> storage being accessed using mmap.)

On the call, I mentioned that CM requires assistance. That assistance could come from the kernel, a userspace daemon, or the NIC/HCA/HFI.


>> Storage and/or persistent devices _may_ need something that 
>> process-to- process model does not. I am trying to get people to 
>> think about the semantics. Is the interface expressive enough?
> 
> As defined, my guess is no.  But based on my current understanding, the framework seems sufficient to go in one of several possible directions.
> 
> Maybe it makes sense to define ideal interfaces for storage, ignoring 
> all other interfaces in the process.  Then see if the resulting API 
> maps well with libfabric/interface-X, rather than working backwards.  
> (And maybe that's what the DS/DA WG is doing.)
> 
> - Sean

I think that we are stuck and this is my poor attempt to add some grease. ;-)

Scott

_______________________________________________
ofiwg mailing list
ofiwg at lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ofiwg