[ofiwg] OFIWG 4/21/2026 Minutes

[Xiong, Jianxin]

4/21/2026

Participants
----------------

Alexander Oganezov (HPE)
Alexia Ingerson (Intel)
Howard Pritchard (LANL)
James Swaro [HPE]
Jerome Soumagne (HPE)
Jianxin Xiong (Working Group Chair) (Intel)
Seth Zegelstein (AWS)
Rajalaxmi (Intel)


Notes (Generated by AI)
--------------------------------

** Overview **

The meeting focused on advancing the design of an accelerator-initiated communication API to enable GPUs and other accelerators to directly manage communication with network interface cards without CPU involvement. Key participants included the Working Group Chair, Seth Zegelstein from AWS, and James Swaro from HPE, who contributed technical insights and raised important considerations.

Discussions covered the history and transfer of the specification from the Ultra Ethernet Consortium to the OpenFabrics Alliance, emphasizing broader community engagement. The group detailed the API design involving shared work and completion queues between accelerators and NICs, with the CPU responsible for initial setup and the accelerator handling the critical data path.

Technical challenges were addressed, including the necessity of both completion queues and counters for error handling, the complexity of implementing provider-specific details, and accommodating diverse accelerator threading models, especially GPUs with massive parallelism. Seth Zegelstein highlighted practical concerns such as static linking of CUDA kernels and the need for flexible memory allocation strategies on the host side.

The Chair presented approaches for exporting communication objects to accelerators, enabling kernel-level access to communication resources. The group recognized the importance of defining clear use cases, with MOE applications suggested as a relevant example to guide development.

Overall, the meeting concluded with agreement on the complexity of the task, the need for ongoing experimentation and refinement, and a commitment to continue collaborative discussions to evolve the API design effectively.


** Detailed Summary **

* Accelerator Initiated Communication

The meeting began with an introduction to the accelerator-initiated communication API, aiming to enable GPUs and other accelerators to directly initiate communication without CPU involvement. This approach is significant as it allows the accelerator to manage communication setup and data transfer directly with the network interface card (NIC), improving efficiency and reducing CPU overhead. The current state involves defining this API to facilitate such direct communication, building on concepts like GPU Direct Async Kernel Initiated (GDA-KI).

- Working Group Chair explains the goal of enabling accelerators to directly initiate communication with NICs without CPU involvement.
- The Chair notes that traditional communication involved CPU setup, but the new API aims for GPU direct communication.
- The concept is aligned with Nvidia's GPU Direct Async Kernel Initiated technology.

* Specification History and Ownership Transfer

The specification for the accelerator API started under the Ultra Ethernet Consortium (UEC) software working group nearly two years ago and reached a technical ready version by the end of last year. However, recognizing that the API is not tightly tied to UEC, the specification ownership was transferred to the OpenFabrics Alliance (OFA) in January to allow broader discussion and development. This transfer is important for aligning the API with the OFA's fabric interface framework and fostering wider community involvement.

- Working Group Chair details the spec's origin under UEC and its technical readiness status.
- The software working group voted to transfer the spec to OFA for broader applicability.
- The transfer occurred in January, enabling open discussions outside the UEC community.

* API Design and Queue Management

The API design involves shifting the role of posting work requests and handling completion queues from the CPU to the accelerator. The accelerator shares work and completion queues with the NIC, posting requests directly and ringing doorbells to notify the NIC. Memory regions and counters are also part of the design, with counters being simpler accumulators accessible by the accelerator. This design is crucial for enabling efficient accelerator-driven communication while maintaining CPU responsibility for initial resource setup.

- Working Group Chair describes the shared work and completion queues between accelerator and NIC.
- Accelerator posts work requests and rings doorbells to trigger NIC processing.
- Counters are simpler than completion queues and accessible by the accelerator.
- CPU remains responsible for initial setup of communication resources.

* Discussion on Counters and Completion Queues

A detailed discussion occurred regarding the role of counters and completion queues (CQ) in the API. Seth Zegelstein from AWS clarified that enabling an endpoint requires both a CQ and a counter, and that errors are reported via the CQ. The Chair acknowledged the simplification in the diagram but confirmed the necessity of both components. This exchange highlights the complexity of error handling and resource management in the API design.

- Seth Zegelstein clarifies that both CQ and counters are required to enable an endpoint.
- Errors are reported on the CQ, necessitating its presence alongside counters.
- Working Group Chair agrees and notes the diagram's simplification.

* Implementation Specifics and Provider Details

The Chair emphasized that while the API can be standardized, the implementation will be provider-specific, depending on the accelerator and NIC hardware. Details such as queue formats, doorbell mechanisms, and synchronization are left to the provider implementations. This approach balances the need for a common interface with the realities of diverse hardware architectures.

- Working Group Chair states API standardization is possible but implementations are hardware-specific.
- Provider implementations will handle queue formats and synchronization details.
- This allows flexibility while maintaining a common API.

* Accelerator Side API Complexity and Threading Models

Participants discussed the complexity of the accelerator-side API, especially considering GPU threading models with thousands of threads potentially posting work requests simultaneously. Seth Zegelstein and James Swaro highlighted challenges in supporting parallelism and synchronization, noting that simplified or specialized APIs might be necessary. The discussion acknowledged that the API must accommodate diverse accelerator architectures beyond GPUs.

- Seth Zegelstein notes GPUs can have thousands of threads posting work simultaneously, complicating serialization.
- James Swaro emphasizes the need to consider various accelerator architectures and threading models.
- The Chair acknowledges the need for further investigation into accelerator-side API design.

* Kernel Linking and API Usage on Accelerators

Seth Zegelstein raised concerns about static linking requirements for CUDA kernels, which may impact traditional dynamic library approaches for the accelerator API. The Chair responded that kernel code could be provided as headers for static inclusion, maintaining a consistent programming model. This discussion is important for practical deployment of the API on accelerator platforms.

- Seth Zegelstein points out CUDA kernels often require static linking, complicating dynamic library use.
- Working Group Chair suggests providing kernel code as headers for static inclusion.
- This approach aims to maintain a consistent API usage model on accelerators.

* Host Side API and Memory Allocation Strategies

The meeting covered the host-side API design for setting up communication objects with memory accessible by accelerators. A user-defined allocation method allows allocation of device-accessible memory, and an import function enables importing externally allocated memory such as doorbell registers. Alternatives include standardizing allocation interfaces or passing user-allocated memory blocks. This flexibility is key to supporting diverse hardware and usage scenarios.

- Working Group Chair explains user-defined allocation and import methods for device-accessible memory.
- Importing memory is necessary for resources like doorbell registers mapped outside device memory.
- Alternatives include standardizing allocation interfaces or passing user-allocated memory.

* Exporting Communication Objects to Accelerators

After creating communication objects on the host, the API supports exporting these objects as memory blocks containing all necessary information for accelerator access. This enables passing pointers to kernels, facilitating direct accelerator interaction with communication resources. The design supports various usage patterns, including passing multiple endpoints to kernels or using global memory areas.

- Working Group Chair describes exporting communication objects as memory blocks for accelerator access.
- Exported memory includes queue locations, sizes, and formats needed by the accelerator.
- Pointers to these blocks can be passed to kernels or stored in global memory for flexible usage.

* Use Cases and Future Work

The group discussed the need for clear use cases to guide API development, with Seth Zegelstein suggesting MOE (Mixture of Experts) applications as a starting point. The Chair noted ongoing experiments and implementations on specific hardware to inform design revisions. The discussion acknowledged the complexity and long-term nature of the work, emphasizing collaboration and iterative refinement.

- Seth Zegelstein recommends MOE applications as practical use cases for the API.
- Working Group Chair mentions ongoing experiments to gather implementation insights.
- James Swaro stresses the importance of a clear high-level design goal for accelerators.
- The group agrees on the need for continued discussion and iterative development.

Jianxin Xiong
Fabric Software
Intel Corporation
Jianxin.xiong at intel.com