Reformulating Regression Test Suite Optimization using Quantum Annealing -- an Empirical Study

TL;DR

This paper reframes regression test suite optimization as a $QUBO$ problem solvable by quantum annealing and introduces SelectQA, a method that balances execution cost and fault-detection effectiveness (and, in a two-objective variant, cost and past failure rate). It compares SelectQA against traditional TCS algorithms (e.g., DIV-GA, Additional Greedy) and a prior quantum approach BootQA, using both real-world and synthetic benchmarks. Empirically, SelectQA often yields the largest set of non-dominated solutions and, in many cases, dominates BootQA in effectiveness while offering competitive efficiency; BootQA remains the more time-efficient option due to solving smaller subproblems directly on the QPU. Overall, the study demonstrates the potential of quantum annealing hybrids for scalable regression testing and outlines paths toward refined quantum strategies such as QAOA and advanced decomposition techniques.

Abstract

Maintaining software quality is crucial in the dynamic landscape of software development. Regression testing ensures that software works as expected after changes are implemented. However, re-executing all test cases for every modification is often impractical and costly, particularly for large systems. Although very effective, traditional test suite optimization techniques are often impractical in resource-constrained scenarios, as they are computationally expensive. Hence, quantum computing solutions have been developed to improve their efficiency but have shown drawbacks in terms of effectiveness. We propose reformulating the regression test case selection problem to use quantum computation techniques better. Our objectives are (i) to provide more efficient solutions than traditional methods and (ii) to improve the effectiveness of previously proposed quantum-based solutions. We propose SelectQA, a quantum annealing approach that can outperform the quantum-based approach BootQA in terms of effectiveness while obtaining results comparable to those of the classic Additional Greedy and DIV-GA approaches. Regarding efficiency, SelectQA outperforms DIV-GA and has similar results with the Additional Greedy algorithm but is exceeded by BootQA.

Figures (3)

• Figure 1: BootQA bootqa overview
• Figure 2: Additional Greedy performed the best with flex, while SelectQA with grep and sed, and DIV-GA with gzip.
• Figure 3: Costs and failure rates comparisons