Quantum Program Testing Through Commuting Pauli Strings on IBM's Quantum Computers

TL;DR

This work tackles practical barriers to industrial quantum software testing by introducing QOPS, a test framework based on commuting Pauli strings that eliminates the need for explicit inputs and pairs with an expectation-value oracle. By partitioning Pauli strings into commuting families (Pauli families), QOPS drastically reduces the amount of program specification required, while remaining compatible with common error mitigation methods and industrial APIs like IBM's Estimator. The approach yields an oracle Exp that leverages the eigenvalues of Pauli strings and their expectation values to assess test outcomes, and it demonstrates superior fault detection over the state-of-the-art ETO across nearly 195k real quantum programs and three IBM real quantum computers. Empirical results show perfect F1-score, precision, and recall for RQ1, and robust performance with error mitigation on hardware for RQ2, highlighting strong industrial relevance and scalability. The study also discusses integration with other frameworks and outlines future directions toward more efficient test-case generation via search-based methods and deeper fault-type analyses.

Abstract

The most promising applications of quantum computing are centered around solving search and optimization tasks, particularly in fields such as physics simulations, quantum chemistry, and finance. However, the current quantum software testing methods face practical limitations when applied in industrial contexts: (i) they do not apply to quantum programs most relevant to the industry, (ii) they require a full program specification, which is usually not available for these programs, and (iii) they are incompatible with error mitigation methods currently adopted by main industry actors like IBM. To address these challenges, we present QOPS, a novel quantum software testing approach. QOPS introduces a new definition of test cases based on Pauli strings to improve compatibility with different quantum programs. QOPS also introduces a new test oracle that can be directly integrated with industrial APIs such as IBM's Estimator API and can utilize error mitigation methods for testing on real noisy quantum computers. We also leverage the commuting property of Pauli strings to relax the requirement of having complete program specifications, making QOPS practical for testing complex quantum programs in industrial settings. We empirically evaluate QOPS on 194,982 real quantum programs, demonstrating effective performance in test assessment compared to the state-of-the-art with a perfect F1-score, precision, and recall. Furthermore, we validate the industrial applicability of QOPS by assessing its performance on IBM's three real quantum computers, incorporating both industrial and open-source error mitigation methods.

Figures (6)

• Figure 1: Two-qubit entanglement quantum circuit with Z and Y measurement.
• Figure 2: An example compact program specification ($\mathit{PS_{compact}}$) for the entanglement circuit from Figure \ref{['fig:twoqubit']}, showing the outputs of two Pauli strings ZX and YI. ZX belongs to a commuting family consisting of (XY, ZX, YZ) and YI belongs to a commuting family consisting of (IY, YY, YI).
• Figure 3: Example comparison of traditional test case definition (Binary, Quantum) with Pauli Strings.
• Figure 4: Overview of $\mathtt{QOPS}$: The tuple $(\mathit{F}\xspace, \mathit{EV}\xspace, \mathit{M}\xspace)$ includes the Pauli family ($\mathit{F}$), the eigenvalues ($\mathit{EV}$) of the Pauli strings in family $\mathit{F}$, and the Specified outcome ($\mathit{M}$) of a single Pauli string in $\mathit{F}$ based on a compact program specification ($\mathit{PS_{compact}}$).
• Figure 5: An example of IBM Estimator API to execute the test case with inbuilt error mitigation enabled.