[ofiwg] FW: HOTI23 notification for paper 28

[Paul Grun]

Good news!  Thanks to Bob Russell and Sean Hefty who did the lions' share of the writing on this.  The hard part now is going to be slimming it down to fit in the short form format.
-Paul

-----Original Message-----
From: HOTI23 [mailto:hoti23 at easychair.org] 
Sent: Monday, June 15, 2015 4:52 PM
To: Paul Grun
Subject: HOTI23 notification for paper 28

Dear Paul, 

Congratulations, your paper "A Brief Introduction to the OpenFabrics Interfaces - A New Network API for Maximizing High Performance Application Efficiency" has been accepted as a short paper for Hot Interconnects 2015 to be held in Santa Clara, CA, August 26-27th. 

The reviews of your paper are attached to this email. Please take the feedback in the reviews into account when preparing your final camera ready version. Author instructions and the camera-ready due date will be sent in the near future. 

An author of the paper must register and present your paper at Hot Interconnects for it to be included in the IEEE eXplore indexed proceedings. The page limit for short papers is 4 pages, although up to 2 additional pages may be purchased for a nominal fee through the registration website. Registration for HotI is expected to openly shortly, please check the HotI website (www.hoti.org) late this week. 

Thank you for submitting your work to Hot Interconnects, we look forward to seeing your work at this year's conference. 

Regards,
Ryan Grant and Ada Gavrilovska
HotI TPC Chairs


----------------------- REVIEW 1 ---------------------
PAPER: 28
TITLE: A Brief Introduction to the OpenFabrics Interfaces - A New Network API for Maximizing High Performance Application Efficiency
AUTHORS: Paul Grun, Sean Hefty, Sayantan Sur, Dave Goodell, Robert Russell, Howard Pritchard and Jeffrey Squyres

OVERALL EVALUATION: 2 (accept)
REVIEWER'S CONFIDENCE: 4 (high)
Relevance to HotI: 5 (excellent)
Technical Soundness: 4 (good)
Presentation Quality: 3 (fair)

----------- REVIEW -----------
While this paper is very relevant to the conference and very well describes an important interface for the community, I would better like it as a keynote and/or a tutorial. Still, I think accepting this paper will add to the meat of the conference. That said, given the available space, I would like to see more discussion of the actual API and some insightful code examples / snippets that could show use cases for various ULPs such as MPI / RMA. A more detailed discussion on new developments in memory regions and registration would also be helpful. Moreover, if available, comparative performance results for a given benchmark under both verbs and OFI would make this a more complete presentation.


----------------------- REVIEW 2 ---------------------
PAPER: 28
TITLE: A Brief Introduction to the OpenFabrics Interfaces - A New Network API for Maximizing High Performance Application Efficiency
AUTHORS: Paul Grun, Sean Hefty, Sayantan Sur, Dave Goodell, Robert Russell, Howard Pritchard and Jeffrey Squyres

OVERALL EVALUATION: -2 (reject)
REVIEWER'S CONFIDENCE: 4 (high)
Relevance to HotI: 4 (good)
Technical Soundness: 2 (poor)
Presentation Quality: 5 (excellent)

----------- REVIEW -----------
This paper describes the architecture of the new user space library libfabric. 
It suggests it is a better implementation than OFA libibverbs and outlines the set of objects used. 
Some details about implementation plan are also provided.

The information may be relevant to some of the HOTI participated. 
The paper is very well written and is easy to follow. 

The problem is that this is not a regular paper. It is unclear what the hypothesis is (maybe the need for such work?), there are no clear arguments why some claims are true and there is no evaluation to support it. 

The paper does deliver to its promise set by the abstract: brief motivation, gathered requirements, overview of APIs and status. 

The claim for novelty of the requirements gathering process is questionable.


----------------------- REVIEW 3 ---------------------
PAPER: 28
TITLE: A Brief Introduction to the OpenFabrics Interfaces - A New Network API for Maximizing High Performance Application Efficiency
AUTHORS: Paul Grun, Sean Hefty, Sayantan Sur, Dave Goodell, Robert Russell, Howard Pritchard and Jeffrey Squyres

OVERALL EVALUATION: 1 (weak accept)
REVIEWER'S CONFIDENCE: 5 (expert)
Relevance to HotI: 5 (excellent)
Technical Soundness: 3 (fair)
Presentation Quality: 5 (excellent)

----------- REVIEW -----------
The authors present the OpenFabrics Interfaces (OFI), a set of communication service APIs for high performance computing, focusing on the first such interface in this set, known as Libfabric. Motivation for this new API is driven by lessons learned during the rapid evolution of the IBTA verbs, iWARP, and RoCE specifications and via input from a growing number of applications using these technologies. The paper would benefit by providing a few exemplars as to what lessons were learned during this process. Are we to take at face value that the new libfabric API is better than these previous efforts? Examples showing how libfabrics addressed at least one fundamental deficiency would strengthen the paper. 

The Libfabric service decomposition seems reasonable given modern interconnect capabilities. Providing applications with a variety of interfaces based on application requirements and hardware capabilities (completion queues or counters, wait sets or poll sets) seems to be a reasonable design point given the breadth of applications and hardware that libfabrics will need to support. 


Overall, the authors present a reasonable set of APIs given modern network capabilities and application requirements, but my two main issue that I have with the paper is the lack of any quantifiable metrics on how well they have achieved their goals and the lack of a clear novel contribution to the field of high performance networking. The paper would benefit from some concrete metrics either in terms of performance (latency, bandwidth, scalability) or a better mapping to application requirements (lines of code, abstraction simplification, new capabilities not previously provided) when compared to other networking APIs such as verbs, sockets, psm, cci, portals, etc..


----------------------- REVIEW 4 ---------------------
PAPER: 28
TITLE: A Brief Introduction to the OpenFabrics Interfaces - A New Network API for Maximizing High Performance Application Efficiency
AUTHORS: Paul Grun, Sean Hefty, Sayantan Sur, Dave Goodell, Robert Russell, Howard Pritchard and Jeffrey Squyres

OVERALL EVALUATION: 0 (borderline paper) REVIEWER'S CONFIDENCE: 5 (expert) Relevance to HotI: 5 (excellent) Technical Soundness: 4 (good) Presentation Quality: 4 (good)

----------- REVIEW -----------
This paper describes the OpenFabrics Interfaces effort. The paper begins by motivating the need for a new API for providing networking services for high-performance computing applications and services. OFI or libfabrics is an activity under the OpenFabrics Alliance and aims to reduce the semantic gap between network service providers and consumers. The paper describes the four different components of the architecture and the different services that are provided. OFI is based on an object model and each object has a set of functions. The paper provides details on these objects and the different functions of each. The paper touches on interesting capabilities, such as an address vector that can be used to provide a logical to physical translation of network addresses to minimize the cost of a lookup. The paper finishes by summarizing the current state of the implementation.

The biggest issue with this paper is that it simply provides an overview of the OFI effort. While there does appear to be some novelty in the approach, it is hard to determine if or how well libfabrics meets its requirements. The paper does mention that the first release currently supports several different transports, so it is disappointing that there is no preliminary performance evaluation in the paper.