Platform-Agnostic Modular Architecture for Quantum Benchmarking

TL;DR

This work presents a platform-agnostic modular architecture that decouples problem generation, circuit execution, and results analysis to unify quantum benchmarking across disparate frameworks. By integrating with pyGSTi for advanced circuit analysis and CUDA-Q for multi-GPU HPC simulation, the approach validates cross-framework interoperability while enabling dynamic circuit variants and a Quantum Reinforcement Learning benchmark. The kernel-based, runtime-loading design, along with a get_circuits capability, preserves optimization flexibility and simplifies expansion to new APIs. Overall, the architecture reduces ecosystem fragmentation and fosters a more cohesive ecosystem for evaluating quantum devices and software stacks with standardized interfaces and metrics.

Abstract

We present a platform-agnostic modular architecture that addresses the increasingly fragmented landscape of quantum computing benchmarking by decoupling problem generation, circuit execution, and results analysis into independent, interoperable components. Supporting over 20 benchmark variants ranging from simple algorithmic tests like Bernstein-Vazirani to complex Hamiltonian simulation with observable calculations, the system integrates with multiple circuit generation APIs (Qiskit, CUDA-Q, Cirq) and enables diverse workflows. We validate the architecture through successful integration with Sandia's $\textit{pyGSTi}$ for advanced circuit analysis and CUDA-Q for multi-GPU HPC simulations. Extensibility of the system is demonstrated by implementing dynamic circuit variants of existing benchmarks and a new quantum reinforcement learning benchmark, which become readily available across multiple execution and analysis modes. Our primary contribution is identifying and formalizing modular interfaces that enable interoperability between incompatible benchmarking frameworks, demonstrating that standardized interfaces reduce ecosystem fragmentation while preserving optimization flexibility. This architecture has been developed as a key enhancement to the continually evolving QED-C Application-Oriented Performance Benchmarks for Quantum Computing suite.

Figures (17)

• Figure 1: QED-C Framework Modularization. The new QED-C architecture modularizes benchmarks into three components: problem generation, circuit execution, and results analysis. Each module consists of easy-to-access methods or variables for utilization.
• Figure 2: Static vs. Dynamic Circuit. (a) In the static circuit, the Hadamard and controlled-$R_{0}$ gates are applied before the final measurement. (b) In the dynamic variant, the first qubit is measured immediately after the Hadamard, and the classical output is used to conditionally trigger the single qubit $R_{0}$ rotation. Finally, the second qubit is measured.
• Figure 3: Illustration of Reinforcement Learning. Figure showing the interaction between agent and environment, along with policy and value functions (e.g., Q-value table).
• Figure 4: QED-C and pyGSTi Integration. Two benchmarking process flows through QED-C's suite and Sandia QPL's pyGSTi framework. The upper workflow shows the simulation of a benchmark from the QED-C's suite using pyGSTi's sophisticated noise models. In this case, pyGSTi simply replaces the execution of the QED-C's circuit on real hardware or a provider's built-in simulators. The upper workflow shows using pyGSTi to convert a QED-C benchmark into a scalable benchmark for measuring process fidelity. In this case, pyGSTi converts the circuits created by the QED-C's suite into other circuits (using, e.g., mirror circuit fidelity estimation proctor2022establishing) that are then executed (on real hardware or a simulator). pyGSTi then processes the data to produce estimates of circuit process fidelities, and can create a variety of performance summary plots.
• Figure 5: Static vs. Dynamic IQFT Circuits. (a) In the static circuit, the Hadamard and controlled-$R_z$ gates are applied in a fixed sequence with final measurement at the end. (b) In the dynamic variant, each qubit is measured immediately after the Hadamard, and the measurement result determines whether to apply the subsequent $R_z$ rotations.