METFORD -- Mutation tEsTing Framework fOR anDroid

TL;DR

METFORD tackles the high cost of Android mutation testing by implementing both traditional and mutant-schemata strategies within a Kadabra-based framework. It encodes all mutants in a single metaprogram (schemata) or in separate mutant variants (traditional), enabling direct, apples-to-apples comparison. Empirical results across six real Android apps show that mutant schemata reduce generation time by about 8.5%, shrink disk usage by roughly 99.8%, and cut overall execution time by about 6.45%, with an 8.18% lower carbon footprint. The work demonstrates the practicality of schemata-based mutation testing for mobile apps while highlighting the need for richer Android UI test suites to improve fault detection.

Abstract

Mutation testing may be used to guide test case generation and as a technique to assess the quality of test suites. Despite being used frequently, mutation testing is not so commonly applied in the mobile world. One critical challenge in mutation testing is dealing with its computational cost. Generating mutants, running test cases over each mutant, and analyzing the results may require significant time and resources. This research aims to contribute to reducing Android mutation testing costs. It implements mutation testing operators (traditional and Android-specific) according to mutant schemata (implementing multiple mutants into a single code file). It also describes an Android mutation testing framework developed to execute test cases and determine mutation scores. Additional mutation operators can be implemented in JavaScript and easily integrated into the framework. The overall approach is validated through case studies showing that mutant schemata have advantages over the traditional mutation strategy (one file per mutant). The results show mutant schemata overcome traditional mutation in all evaluated aspects with no additional cost: it takes 8.50% less time for mutant generation, requires 99.78% less disk space, and runs, on average, 6.45% faster than traditional mutation. Moreover, considering sustainability metrics, mutant schemata have 8,18% less carbon footprint than traditional strategy.

Figures (16)

• Figure 1: Example of how applying directly the schemata strategy can introduce compilation errors.
• Figure 2: Example of declaration transformation to support mutant schemata strategy.
• Figure 3: Original code (from project OmniNotes).
• Figure 4: Code after applying the schemata strategy (from project OmniNotes).
• Figure 5: Method that fetches the property that controls the executed mutation.