MMT: Mutation Testing of Java Bytecode with Model Transformation -- An Illustrative Demonstration

TL;DR

This paper addresses the limitations of traditional Java bytecode mutation tools, which largely cover basic operator mutations and overlook advanced object-oriented and API-related mutations. It proposes a model-driven approach, MMT, that represents Java bytecode with the Mod-BEAM EMF metamodel and implements mutation operators as Henshin transformations, enabling 68 advanced operators. The tool emphasizes syntactic and semantic safety via OCL constraints and provides an Eclipse-based GUI for managing mutants and extending operators. The work demonstrates improved mutation coverage and identifies test-suite weaknesses, with plans for broader benchmarking and performance analysis against existing tools.

Abstract

Mutation testing is an approach to check the robustness of test suites. The program code is slightly changed by mutations to inject errors. A test suite is robust enough if it finds such errors. Tools for mutation testing usually integrate sets of mutation operators such as, for example, swapping arithmetic operators; modern tools typically work with compiled code such as Java bytecode. In this case, the mutations must be defined in such a way that the mutated program still can be loaded and executed. The results of mutation tests depend directly on the possible mutations. More advanced mutations and even domain-specific mutations can pose another challenge to the test suite. Since extending the classical approaches to more complex mutations is not well supported and is difficult, we propose a model-driven approach where mutations of Java bytecode can be flexibly defined by model transformation. The corresponding tool called MMT has been extended with advanced mutation operators for modifying object-oriented structures, Java-specific properties and method calls of APIs, making it the only mutation testing tool for Java bytecode that supports such mutations.

Figures (24)

• Figure 1: Example code to which mutation testing can be applied. When the code in the red box is removed, the test still succeeds.
• Figure 2: Details reported on a specific mutant, indicating a live mutant.
• Figure 3: Mutation operator for deleting an overriding method specified as Henshin transformation rule.
• Figure 4: A snippet from the instance model in abstract graphical syntax corresponding to the source code in Fig. \ref{['fig:init-example']}, showing the match of the mutation operator in Fig. \ref{['fig:deleteOverridingMethod']}.
• Figure 5: Java source code before mutation. The last field initialization in the constructor of the class "User" is to be deleted.