Coverage Goal Selector for Combining Multiple Criteria in Search-Based Unit Test Generation

TL;DR

The paper addresses the problem that combining multiple coverage criteria in search-based unit test generation can inflate the optimization landscape and reduce individual criterion performance. It introduces smart selection, a three-step approach that groups criteria by coverage correlation, selects representative criteria to guide SBST, and uses subsumption-based subsets to preserve properties of unselected goals. The approach is integrated with EvoSuite and evaluated across WS, MOSA, and DynaMOSA on 400 Java classes, showing substantial improvements over the original criterion combination, especially for large classes, and demonstrating robust fault-detection gains with MOSA. The findings highlight the benefit of reducing optimization objectives while maintaining coverage properties, with practical impact on improving test generation efficiency and fault detection in real-world Java projects.

Abstract

Unit testing is critical to the software development process, ensuring the correctness of basic programming units in a program (e.g., a method). Search-based software testing (SBST) is an automated approach to generating test cases. SBST generates test cases with genetic algorithms by specifying the coverage criterion (e.g., branch coverage). However, a good test suite must have different properties, which cannot be captured using an individual coverage criterion. Therefore, the state-of-the-art approach combines multiple criteria to generate test cases. Since combining multiple coverage criteria brings multiple objectives for optimization, it hurts the test suites' coverage for certain criteria compared with using the single criterion. To cope with this problem, we propose a novel approach named \textbf{smart selection}. Based on the coverage correlations among criteria and the subsumption relationships among coverage goals, smart selection selects a subset of coverage goals to reduce the number of optimization objectives and avoid missing any properties of all criteria. We conduct experiments to evaluate smart selection on $400$ Java classes with three state-of-the-art genetic algorithms under the $2$-minute budget. On average, smart selection outperforms combining all goals on $65.1\%$ of the classes having significant differences between the two approaches. Secondly, we conduct experiments to verify our assumptions about coverage criteria relationships. Furthermore, we assess the coverage performance of smart selection under varying budgets of $5$, $8$, and $10$ minutes and explore its effect on bug detection, confirming the advantage of smart selection over combining all goals.

Figures (17)

• Figure 1: Partial Data of Coverage Gaps
• Figure 2: Overview of Unit Tests Generation in EvoSuite
• Figure 3: Overview of Smart Selection
• Figure 4: Significant case summary of smart selection and the original combination with WS
• Figure 5: Significant case summary of smart selection and the original combination with MOSA