Optimizing Metamorphic Testing: Prioritizing Relations Through Execution Profile Dissimilarity

TL;DR

This work proposes a statement centrality-based prioritization approach that leverages diversity in the execution profiles of source and follow-up test cases that improves fault detection effectiveness, reduces fault detection time, and enhances testing efficiency.

Abstract

An oracle determines whether the output of a program for executed test cases is correct. For machine learning programs, such an oracle is often unavailable or impractical to apply. Metamorphic testing addresses this by using metamorphic relations (MRs), which are essential properties of the software under test, to verify program correctness. Prioritizing MRs enhances fault detection effectiveness and improves testing efficiency. However, fault-based MR prioritization is costly, and code coverage-based approaches often yield inaccurate results. To address these challenges, we propose a statement centrality-based prioritization approach that leverages diversity in the execution profiles of source and follow-up test cases. Our approach improves fault detection effectiveness by up to 31% compared to code coverage-based methods and reduces fault detection time by 29% compared to random MR execution. Overall, it increases the average fault detection rate, reduces fault detection time, and enhances testing efficiency.

Figures (3)

• Figure 1: Example to show statements affected
• Figure 2: Statement Impacted due to a affected statement
• Figure 3: Fault detection effectiveness for BBMap, LingPipe, Linear Regression and Naive Bayes