QEF: Reproducible and Exploratory Quantum Software Experiments

TL;DR

The paper addresses the challenge of conducting reproducible, hypothesis-driven experiments for hybrid quantum-classical algorithms in the NISQ era and beyond. It introduces the Quantum Experiment Framework (QEF), a lightweight, descriptor-driven platform that expands experiments into Cartesian products of variants, runs large sweeps asynchronously, reuses parameters to speed up exploration, and stores results in tidy CSV format for easy analysis. A literature survey motivates the design by identifying common experiment parameters and metrics, and a Max-Cut with QAOA case study demonstrates precise reproducibility and flexible exploratory analysis. QEF aims to lower the barriers to empirical quantum research by replacing ad-hoc scripting with a coherent, extensible workflow that supports both current NISQ devices and future fault-tolerant quantum systems.

Abstract

Commercially available Noisy Intermediate-Scale Quantum (NISQ) devices now make small hybrid quantum-classical experiments practical, but many tools hide configuration or demand ad-hoc scripting. We introduce the Quantum Experiment Framework (QEF): A lightweight framework designed to support the systematic, hypothesis-driven study of quantum algorithms. Unlike many existing approaches, QEF emphasises iterative, exploratory analysis of evolving experimental strategies rather than exhaustive empirical evaluation of fixed algorithms using predefined quality metrics. The framework's design is informed by a comprehensive review of the literature, identifying principal parameters and measurement practices currently reported in the field. QEF captures all key aspects of quantum software and algorithm experiments through a concise specification that expands into a Cartesian product of variants for controlled large-scale parameter sweeps. This design enables rigorous and systematic evaluation, as well as precise reproducibility. Large sweeps are automatically partitioned into asynchronous jobs across simulators or cloud hardware, and ascertain full hyper-parameter traceability. QEF supports parameter reuse to improve overall experiment runtimes, and collects all metrics and metadata into a form that can be conveniently explored with standard statistical and visualisation software. By combining reproducibility and scalability while avoiding the complexities of full workflow engines, QEF seeks to lower the practical barriers to empirical research on quantum algorithms, whether these are designed for current NISQ devices or future error-corrected quantum systems.

Figures (4)

• Figure 1: Integration of QEF into research workflows. Experiments defined via a structured configuration expand into a Cartesian product of job variants. These jobs are sent to the target hardware, where both, quantum processors and classical computing resources, collaborate to run and evaluate hybrid algorithms. Result data are stored in formats optimised for convenient and dynamic exploratory and visual analysis with standard tools.
• Figure 2: QEF workflow. Experiments are expanded into jobs that run asynchronously. Completed jobs can be retrieved for result analysis.
• Figure 3: Excerpt from the descriptor file for the experiment that was performed in the context of the case study.
• Figure 4: Exploratory analysis of QAOA using QEF