Tensor Algebra Processing Primitives (TAPP): Towards a Standard for Tensor Operations

TL;DR

Tensor contractions are central to AI and quantum simulations, yet the software ecosystem suffers from fragmentation due to the absence of a universal standard. The authors propose Tensor Algebra Processing Primitives (TAPP), a BLAS-like, C-based interface that decouples applications from hardware back-ends while providing a correct reference implementation and a governance-driven ecosystem. The work formalizes tensor contractions, presents a comprehensive interface design, and demonstrates practical integration with TBLIS, cuTENSOR, and the DIRAC quantum chemistry package, highlighting improved portability and reduced dependency risk. By enabling back-end experimentation and stable cross-framework collaboration, TAPP aims to catalyze community-driven evolution of tensor primitives and foster performance portability across architectures.

Abstract

To address the absence of a universal standard interface for tensor operations, we introduce the Tensor Algebra Processing Primitives (TAPP), a C-based interface designed to decouple the application layer from hardware-specific implementations. We provide a mathematical formulation of tensor contractions and a reference implementation to ensure correctness and facilitate the validation of optimized kernels. Developed through community consensus involving academic and industrial stakeholders, TAPP aims to enable performance portability and resolving dependency challenges. The viability of the standard is demonstrated through successful integrations with the TBLIS and cuTENSOR libraries, as well as the DIRAC quantum chemistry package.

Figures (5)

• Figure 1: Relations among the label sets $\mathcal{A}, \mathcal{B}$ and $\mathcal{D}$. Left: Simple contractions (Case 1); right: simple contractions and Hadamard products (Case 2).
• Figure 2: Relations among the label sets $\mathcal{A}, \mathcal{B}$ and $\mathcal{D}$. Left: Repeated indices (Case 3); right: Isolated indices (Case 4).
• Figure 3: Flowchart of the TAPP interface components, with examples of TAPP API signatures. In addition to the dedicated TAPP_attr object, all TAPP objects support key-value semantics, indicated by an orange glow.
• Figure 4: Pseudo-code for the implementation of general tensor contraction (Case 3). Decomposition of the index strings into subsets $\mathcal{P}$, $\mathcal{F_A}$, … and implementation of basic functions such as extent and stride not shown.
• Figure 5: Pseudo-code for helper functions for the implementation of general tensor contraction.