Identifying and Replicating Code Patterns Driving Performance Regressions in Software Systems
Denivan Campos, Luana Martins, Emanuela Guglielmi, Michele Tucci, Daniele Di Pompeo, Simone Scalabrino, Vittorio Cortellessa, Dario Di Nucci, Rocco Oliveto
TL;DR
The paper tackles the challenge of performance regressions by advancing performance mutation testing (PMT) as a practical framework to simulate real-world slowdowns. It proposes mining real-world performance issues to enrich PMT operator catalogs, and it outlines an end-to-end pipeline that includes pattern extraction, operator derivation, and benchmarking on Java projects with JMH microbenchmarks. A gold-standard dataset of performance issues and code pairs is planned, along with public release of generated mutants and analysis scripts. The work aims to improve the detection and prevention of performance regressions and to inform practitioners and researchers with actionable PMT guidance and tooling.
Abstract
Context: Performance regressions negatively impact execution time and memory usage of software systems. Nevertheless, there is a lack of systematic methods to evaluate the effectiveness of performance test suites. Performance mutation testing, which introduces intentional defects (mutants) to measure and enhance fault-detection capabilities, is promising but underexplored. A key challenge is understanding if generated mutants accurately reflect real-world performance issues. Goal: This study evaluates and extends mutation operators for performance testing. Its objectives include (i) collecting existing performance mutation operators, (ii) introducing new operators from real-world code changes that impact performance, and (iii) evaluating these operators on real-world systems to see if they effectively degrade performance. Method: To this aim, we will (i) review the literature to identify performance mutation operators, (ii) conduct a mining study to extract patterns of code changes linked to performance regressions, (iii) propose new mutation operators based on these patterns, and (iv) apply and evaluate the operators to assess their effectiveness in exposing performance degradations. Expected Outcomes: We aim to provide an enriched set of mutation operators for performance testing, helping developers and researchers identify harmful coding practices and design better strategies to detect and prevent performance regressions.