QuCheck: A Property-based Testing Framework for Quantum Programs in Qiskit

TL;DR

QuCheck tackles the hard problem of testing quantum programs by enabling property-based testing within the Qiskit ecosystem. It provides a Python-based framework with flexible input generators, preconditions, and multi-qubit assertions, coupled with statistical analysis and measurement optimization including Holm-Bonferroni correction. In mutation-analysis experiments on five quantum algorithms, three properties yielded high fault-detection scores (0.90–0.92) with low false positives, and QuCheck identified 36.0% more faults while reducing execution time by 81.1% compared with QSharpCheck. The work demonstrates the viability and practicality of property-based testing for quantum systems and highlights improvements in expressivity and efficiency over prior approaches, with implications for broader adoption in quantum software verification.

Abstract

Property-based testing has been previously proposed for quantum programs in Q# with QSharpCheck; however, this implementation was limited in functionality, lacked extensibility, and was evaluated on a narrow range of programs using a single property. To address these limitations, we propose QuCheck, an enhanced property-based testing framework in Qiskit. By leveraging Qiskit and the broader Python ecosystem, QuCheck facilitates property construction, introduces flexible input generators and assertions, and supports expressive preconditions. We assessed its effectiveness through mutation analysis on five quantum programs (2-10 qubits), varying the number of properties, inputs, and measurement shots to assess their impact on fault detection and demonstrate the effectiveness of property-based testing across a range of conditions. Results show a strong positive correlation between the mutation score (a measure of fault detection) and number of properties evaluated, with a moderate negative correlation between the false positive rate and number of measurement shots. Among the most thorough test configurations, those evaluating three properties achieved a mean mutation score ranging from 0.90 to 0.92 across all five algorithms, with the false positive rate between 0 and 0.04. QuCheck identified 36.0% more faults than QSharpCheck, with execution time reduced by 81.1%, despite one false positive. These findings underscore the viability of property-based testing for verifying quantum systems.

Figures (10)

• Figure 1: Major components of the framework.
• Figure 2: Experimental setup.
• Figure 3: Heatmaps illustrating the impact of property count, input size, and measurement quantity on mutation scores. The top row represents the average mutation scores for QMutPy-generated mutants across all algorithms, and the bottom row, the average mutation scores for equivalent mutants.
• Figure 4: Effect of number of properties on mutation score.
• Figure 5: Effect of number of properties on mutation score with 800, 1600, and 3200 measurement shots.