The EU Quantum Flagship's Key Performance Indicators for Quantum Computing

TL;DR

The paper addresses the need for scalable, platform-agnostic performance metrics for quantum processors by introducing four KPIs—the Clifford Volume benchmark, GHZ-state preparation benchmark, Shor period-finding benchmark, and Bell-state quantum error correction benchmark. Each KPI provides a concrete protocol, scoring rule, and reporting standard, with numerical analyses illustrating how performance scales with noise and qubit counts. The suite emphasizes reproducibility, open-source implementations, and fair cross-platform comparisons, aiming to track progress from late-NISQ toward fault-tolerant quantum computation. Together, these benchmarks offer a practical, transparent roadmap for evaluating holistic quantum computer performance across technologies and time.

Abstract

As quantum processors continue to scale in size and complexity, the need for well-defined, reproducible, and technology-agnostic performance metrics becomes increasingly critical. Here we present a suite of scalable quantum computing benchmarks developed as key performance indicators (KPIs) within the EU Quantum Flagship. These proposed benchmarks are designed to assess holistic system performance rather than isolated components, and to remain applicable across both noisy intermediate-scale quantum (NISQ) devices and future fault-tolerant architectures. We introduce four core benchmarks addressing complementary aspects of quantum computing capability: large multi-qubit circuit execution via a Clifford Volume benchmark, scalable multipartite entanglement generation through GHZ-state preparation, a benchmark based on the application of Shor's period-finding subroutine to simple functions, and a protocol quantifying the benefit of quantum error correction using Bell states. Each benchmark is accompanied by clearly specified protocols, reporting standards, and scalable evaluation methods. Together, these KPIs provide a coherent framework for transparent and fair performance assessment across quantum hardware platforms and for tracking progress late-NISQ toward early fault-tolerant quantum computation.

Figures (8)

• Figure 1: Worst-case expectation values of randomly chosen stabilizer operators (blue curve) and non-stabilizer operators (red curve) corresponding to four randomly chosen Clifford unitaries as a function of the qubit number $N$. For the simulation, error probabilities of $p_{2Q}=10^{-3}$ and $p_{m}=10^{-2}$ were assumed for two-qubit gates and for the readout, respectively. Shaded regions represent the statistical uncertainties of $\pm 2\sigma$.
• Figure 2: Clifford Volume benchmark scores for different pairs of error parameters. The horizontal axis corresponds to the two-qubit gate error probability $p_{2Q}$, and the vertical axis to the readout error probability $p_m$. Each cell indicates the largest qubit count $N$ for which the benchmark criteria are satisfied under the corresponding noise configuration.
• Figure 3: Stabilizer-based lower bound on the GHZ-state fidelity, $\hat{F}_{\min}$, as a function of the number of qubits for four different combinations of two-qubit depolarizing error probability $p_{2Q}$ and measurement error probability $p_m$. Solid lines show the estimated fidelity value, while shaded regions indicate the $\pm 3\sigma$ statistical uncertainty due to finite sampling. The horizontal dashed line marks the entanglement threshold $\hat{F}_{\min}=1/2$, above which genuine multipartite entanglement is certified. All expectation values were estimated from $8192$ measurement shots per measurement setting.
• Figure 4: Multipartite entanglement benchmark scores obtained from numerical simulations for different combinations of two-qubit depolarizing error probability $p_{2Q}$ and measurement bit-flip error probability $p_m$. Each cell shows the largest number of qubits $N$ for which the stabilizer-based fidelity lower bound $\hat{F}_{\min}$ remains above the entanglement threshold $1/2$ with $3\sigma$ statistical confidence. The results illustrate the strong dependence of the certifiable GHZ state size on both gate and measurement error rates.
• Figure 5: Shor's period-finding quantum circuit. The quantum circuit uses a total of $3n+c$ qubits: $t=2n+c$ qubits in the control register and $n$ qubits to implement permutation gates. The $U_M$ gate implements a linear permutation specified by the $n\times n$ matrix $M$. Its powers $M^{2^q}$, $q\in \mathbb{N}$ are computed classically and the corresponding quantum gates $U_{M^{2^q}}$ are compiled.