[Ofiwg-mpi] Fwd: OpenFabrics lib fabric effort: next round of feedback

Thu May 1 11:00:23 PDT 2014

Below is an email I BCC'ed to a bunch of MPI community people before we got this listserv, including Sean Hefty's original email and his slides.

I've already received some feedback on Sean's slides; I'll summarize what I've received so far in subsequent emails.  Please feel free to comment / reply with more thoughts to the 3 specific questions asked below.

If it helps, we can certainly have a webex for wider discussion.

Begin forwarded message:

From: "Jeff Squyres (jsquyres)" <jsquyres at cisco.com<mailto:jsquyres at cisco.com>>
Subject: OpenFabrics lib fabric effort: next round of feedback
Date: April 23, 2014 6:45:08 AM EDT
To: "Jeff Squyres (jsquyres)" <jsquyres at cisco.com<mailto:jsquyres at cisco.com>>
Cc: Sean Hefty <sean.hefty at intel.com<mailto:sean.hefty at intel.com>>

MPI Community (BCC'ed -- see below):

Per our prior discussions back in January, the libfabric group is getting farther along in developing fabric-agnostic APIs.  Sean Hefty, CC'ed, is the main technical leader of this effort.

Sean sent an email last week that contains a PPT and a link to man pages for a bunch of proposed APIs.  See his original email, including the PPT and link, below.

This is very early days; these should probably be considered super-early-first-draft-here's-what-we're-thinking-about kinds of APIs.  Many of them are similar to / influenced by the existing Linux verbs API.  Many others are totally new / not anywhere close to anything in Linux verbs.

There's (at least) 3 questions on the table for the MPI community:

0. How do we want to have a discussion about this stuff?  Do we want a listserv to jointly discuss the MPI-specific aspects of these APIs?  (vs. me BCC'ing blastograms to you)  Do we want to have another webex soon?  ...?

** I ask this question because there's a larger / general libfabric API discussion going on that the greater MPI community may or may not care about.  These occur (somewhat) on the OpenFabrics IWG listserv, and mostly on the Tuesday webexes (12pm US Eastern).  Do we want a place/time for MPI-specific aspects of this conversation?

1. As you can see in the PPT, there's some discussion occurring about how to integrate this new effort into the old Linux verbs API.  a) Should all the new APIs just be "extensions" to Linux verbs?  b) Should it be a whole new API?  c) Should the new API include (by value) the old Linux verbs APIs?  d) ...?

** These questions are aimed at a) applications (e.g., MPI implementations) forward-porting to libfabric and b) vendors providing both verbs and libfabric support that can be compatible with each other.

2. What do you think of the *general direction* of these APIs?  Don't yet harp on specific parameters and functions (expressed as function pointers in the PPT, BTW) -- just comment on whether you think the overall direction / high-level ideas are headed in the right direction.  And if not, indicate why not.

-----

For reference, here's who's BCC'ed in no particular order -- anyone missing?  (my mail client decided to drop the BCC lists from the mails I sent back in January/February, so this list may not be complete)

Dave Goodell <dgoodell at cisco.com<mailto:dgoodell at cisco.com>>
Cesare Cant� <cantuc at cisco.com<mailto:cantuc at cisco.com>>
Reese Faucette <rfaucett at cisco.com<mailto:rfaucett at cisco.com>>
Amin Hassani <ahassani at cis.uab.edu<mailto:ahassani at cis.uab.edu>>
Brad Benton <Brad.Benton at amd.com<mailto:Brad.Benton at amd.com>>
Charles J Archer <charles.j.archer at intel.com<mailto:charles.j.archer at intel.com>>
Daniel Holmes <dholmes at epcc.ed.ac.uk<mailto:dholmes at epcc.ed.ac.uk>>
Devendar Bureddy <devendar at mellanox.com<mailto:devendar at mellanox.com>>
Jeff Hammond <jeff.science at gmail.com<mailto:jeff.science at gmail.com>>
Michael Blocksome <blocksom at us.ibm.com<mailto:blocksom at us.ibm.com>>
Michael Raymond <mraymond at sgi.com<mailto:mraymond at sgi.com>>
Nathan Hjelm <hjelmn at lanl.gov<mailto:hjelmn at lanl.gov>>
Pavel Shamis <shamisp at ornl.gov<mailto:shamisp at ornl.gov>>
Ron Brightwell <rbbrigh at sandia.gov<mailto:rbbrigh at sandia.gov>>
Sayantan Sur <sayantan.sur at intel.com<mailto:sayantan.sur at intel.com>>
Ryan Grant <regrant at sandia.gov<mailto:regrant at sandia.gov>>
Torsten Hoefler <htor at inf.ethz.ch<mailto:htor at inf.ethz.ch>>
Chulho Kim <chulho at us.ibm.com<mailto:chulho at us.ibm.com>>
Carl Obert <obert at us.ibm.com<mailto:obert at us.ibm.com>>
Perry Schmidt <pschmidt at us.ibm.com<mailto:pschmidt at us.ibm.com>>
Fab Tillier <ftillier at microsoft.com<mailto:ftillier at microsoft.com>>
Shane/Matthew Farmer <shanefarmer22 at gmail.com<mailto:shanefarmer22 at gmail.com>>
Howard Pritchard <howardp at cray.com<mailto:howardp at cray.com>>
Tony Skjellum <tony at cis.uab.edu<mailto:tony at cis.uab.edu>>

Begin forwarded message:

> From: "Hefty, Sean" <sean.hefty at intel.com<mailto:sean.hefty at intel.com>>
> Subject: [ofiwg] presentation from today's WG meeting
> Date: April 15, 2014 1:22:07 PM EDT
> To: "ofiwg at lists.openfabrics.org<mailto:ofiwg at lists.openfabrics.org>" <ofiwg at lists.openfabrics.org<mailto:ofiwg at lists.openfabrics.org>>
>
> Attached is the presentation that I went over today.  Next week's meeting will solicit responses from anyone regarding the *direction* of the proposal.
>
> As mentioned in today's meeting, the man page formats for the current APIs can be found in the source code or online here:
>
> https://www.openfabrics.org/downloads/OFIWG/API/
>
> This is slightly older documentation and describes the static inline wrapper functions around the operations discussed in the attached slides.  The *details* of the specific function calls is what would need discussion within the working group and the application developers.
>
> For a merged effort, I would anticipate that in some cases the same set of function pointers could be usable between gen1 and gen2 APIs (e.g. msg_ops, rma_ops), but with differently named wrapper functions (e.g. fi_write versus ibv_write).  An example of this was in last week's presentation.  In other cases, functions may not easily apply (e.g. tagged_ops) or only the concepts may be transferable (e.g. optimized poll CQ call).  The CM functionality and their full integration would be an example of calls that evolve from gen 1 to a gen 2.
>
> - Sean
> _______________________________________________
> ofiwg mailing list
> ofiwg at lists.openfabrics.org<mailto:ofiwg at lists.openfabrics.org>
> http://lists.openfabrics.org/mailman/listinfo/ofiwg

--
Jeff Squyres
jsquyres at cisco.com<mailto:jsquyres at cisco.com>
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

--
Jeff Squyres
jsquyres at cisco.com<mailto:jsquyres at cisco.com>
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg-mpi/attachments/20140501/59793397/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 2014-04-14-ofiwg-proposal.pptx
Type: application/vnd.openxmlformats-officedocument.presentationml.presentation
Size: 288039 bytes
Desc: 2014-04-14-ofiwg-proposal.pptx
URL: <http://lists.openfabrics.org/pipermail/ofiwg-mpi/attachments/20140501/59793397/attachment.pptx>