[ofiwg] DS/DA Runtime Model Discussion

[Hefty, Sean]

> Do I understand correctly that, in a nutshell, the proposal is
> that kfabrics becomes semantically richer than current
> kernel verbs or any other kernel network interface, which
> would allow to (efficiently, we hope) abstract from any underlying
> fabrics semantics? Wouldn't that just bloat the interface
> implementation far beyond the point the current kernel verbs
> already is ... or is the idea that code can be pushed 'further down'
> below kfabrics and become vendor specific? Your second figure
> hints at the second approach.

All vendor specific code should be pushed further down, and kept within the vendor code.  Pushing vendor specific code into the apps is either an obscene business model or very poor design. 

> I am not yet sold to the idea that kernel programmers would
> talk to kernel sockets via kfabrics. The kernel socket interface
> is very stable and efficient.

I agree with this unless an app requires different semantics than either stream or datagram based message processing.  Otherwise, moving protocol that would be duplicated by multiple apps into a single implementation would be beneficial.

> GPFS deliberately chose to only use user level RDMA interfaces for
> block transfers and metadata operations. So, with that, it does not
> depend on kernel RDMA interfaces, and switching from verbs
> to libfabrics would just mean the pain of some coding effort w/o any
> functional benefit btw.

Verbs implies a very specific implementation, data ordering, and completion ordering semantics.  If high performance over non-verbs hardware or non-IB fabrics isn't a concern, then I agree, there's no functional benefit.