[ofiwg] OFIWG 5/5/2026 Minutes

[Xiong, Jianxin]

5/5/2026

Participants
Bob Cernohous (Cornelis Networks)
Jerome Soumagne (HPE)
Jianxin Xiong (Working Group Chair) (Intel)
Ken Raffenetti (Cornelis)
Sean Pollard [HPE]
Shi Jin (AWS)
Stephen Oost (Intel)
Zach Dworkin (Intel)
Seth Zegelstein (AWS)
Rajalaxmi (Intel)

Overview
The meeting, led by the Working Group Chair with active contributions from AWS representatives Shi Jin and Seth Zegelstein, centers on advancing the Accelerator API proposal to enable accelerator-initiated communication in fabric environments. The Chair outlines the API's dual host and device components, emphasizing memory allocation strategies and exporting host objects for device access. The group debates the design of a simplified accelerator-side API consolidating multiple operations into a single function, with considerations for language-specific device function semantics raised by Shi Jin.

Threading and concurrency emerge as critical topics, with Seth and Shi Jin highlighting the need to support multiple concurrent threads on accelerators to leverage parallelism effectively. The Chair proposes API extensions to manage thread groups and indices, drawing parallels to NVIDIA's threading models. Capability discovery mechanisms are discussed to handle unsupported operations gracefully, favoring early detection during host setup.

The participants agree on maintaining a clear separation between host and device APIs to avoid complexity. Completion queue and counter usage are examined, with suggestions to restrict concurrent access or employ per-operation counters to ensure synchronization. The Chair commits to refining the API design with a focus on multi-threading performance and invites ongoing feedback through the working group's discussion channels.

Detailed Summary

ACCELERATOR API Proposal Overview
The Working Group Chair revisits the ACCELERATOR API proposal, focusing on defining APIs at the fabric layer to enable communication initiated from accelerators like GPUs. The API consists of host-side functions for environment and resource setup and device-side functions callable from compute kernels. Key aspects include creating objects such as endpoints and queues with memory accessible by the device, and exporting host objects for device access.

  *   Working Group Chair explains the API enables accelerator-initiated communication.

  *   Chair details host-side setup and device-side callable functions.

  *   Chair notes the need to allocate device-accessible memory for queues and counters.

  *   Chair describes exporting host objects to device-accessible memory blocks.

Device Memory Allocation and Object Export
The discussion covers memory allocation strategies for device-accessible objects, including allocating memory directly on the device or mapping host memory accessible by the device. The proposal includes functions to import external memory and modify object attributes to include device and interface identifiers. Export functions enable host objects to be accessible from the accelerator side.

  *   Chair proposes functions for device memory allocation and external memory import.

  *   Chair emphasizes modifying object attributes to include device and interface info.

  *   Chair explains exporting host objects to device-accessible memory for accelerator use.

Simplified Accelerator-Side API Design
The group considers a simplified API for accelerator-side operations, consolidating send, receive, read, write, and atomic functions into a single core function with similar signatures to host-side APIs but adapted for device endpoints. The design excludes support for IO vectors and inject calls, reflecting common patterns in existing GPU-initiated communication APIs.

  *   Chair suggests consolidating multiple operations into a single accelerator-side function.

  *   Chair notes the simplified API excludes IO vector and inject call support.

  *   Chair observes existing GPU communication APIs typically use single-buffer processing.

Device Function Signature and Language Support
Shi Jin (AWS) questions how the accelerator API functions will be defined across different programming languages like CUDA, which require specific device function semantics. The Chair responds that while the basic function signature can remain consistent, the exact syntax and keywords will depend on the programming language, with providers implementing language-specific headers or templates.

  *   Shi Jin (AWS) asks about device function semantics for different accelerator languages.

  *   Chair explains the core signature remains but syntax varies by language.

  *   Chair mentions providers may use language-specific headers or templates for implementation.

Threading Model for Accelerator APIs
Seth Zegelstein (AWS) raises concerns about threading models, questioning if the APIs assume single-threaded use and how concurrency is handled. The Chair acknowledges the need to discuss threading models, noting existing host-side models like thread safe, thread domain, and thread completion. Shi Jin emphasizes the importance of supporting concurrent thread calls on accelerators to leverage parallelism.

  *   Seth Zegelstein (AWS) questions threading assumptions and concurrency support.

  *   Chair notes existing host-side threading models and the need to adapt for accelerators.

  *   Shi Jin (AWS) stresses the advantage of concurrent thread execution on accelerators.

Handling Multiple Threads and Work Queues
The Chair discusses implementation strategies for multiple threads posting to the same or multiple work queues, suggesting dividing work queue entries among threads and updating doorbells collectively. They propose adding parameters to the API to indicate group size and thread index to support concurrent operations. Seth references NVIDIA's block, warp, and thread APIs as examples of different parallelism levels.

  *   Chair explains dividing work queue entries among multiple threads for concurrency.

  *   Chair proposes API parameters for group size and thread index to manage concurrency.

  *   Seth Zegelstein (AWS) references NVIDIA's block, warp, and thread APIs for parallelism levels.

API Capability Discovery and Unsupported Operations
The group discusses how providers indicate supported operations via capability bits during setup. If a provider does not support certain operations like send or receive, it can return 'not supported' errors. The Chair suggests early discovery of capabilities at host setup to avoid runtime errors. Device-side error handling is limited, so capability querying is preferred.

  *   Chair recommends capability querying during host setup to identify supported operations.

  *   Seth Zegelstein (AWS) asks about handling unsupported operations and error returns.

  *   Chair notes device-side error handling is limited, favoring early capability discovery.

Separation of Host and Device Side APIs
Seth Zegelstein (AWS) questions whether device-side APIs can be called from the host, concluding that mixing host and device endpoints is discouraged due to complexity. The Chair agrees that mixing is messy and prefers clear separation between host and accelerator endpoints. They acknowledge that while theoretically possible, cross-calling is not planned.

  *   Seth Zegelstein (AWS) advises against calling device APIs from the host side.

  *   Chair concurs that mixing host and device endpoints is complex and undesirable.

  *   Both agree on maintaining clear separation between host and accelerator APIs.

Completion Queue and Counter Usage
The Chair discusses limitations on multiple threads accessing the same completion queue and counters, suggesting usage restrictions or design alternatives. Some APIs separate counters per operation type to synchronize completions more granularly. The group considers these design choices to balance concurrency and correctness.

  *   Chair notes multiple threads should not concurrently read the same completion queue or counter.

  *   Chair mentions alternative designs with separate counters per operation type.
Discussion focuses on balancing concurrency with synchronization correctness.

Jianxin Xiong
Fabric Software
Intel Corporation
Jianxin.xiong at intel.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/ofiwg/attachments/20260505/eedb99f4/attachment-0001.htm>