Search-based Software Testing Driven by Domain Knowledge: Reflections and New Perspectives

TL;DR

This paper addresses the gap where SBST lacks engineers' domain knowledge in CPS testing. It surveys and analyzes SBST frameworks that integrate domain knowledge, focusing on ATheNA and Hecate as concrete instantiations implemented for Simulink. Through evaluation on ARCH benchmarks and automotive/medical case studies, the paper shows that domain-knowledge-guided SBST can outperform traditional SBST and uncover failures that would be missed otherwise, while highlighting context-driven evaluation as essential. It then proposes a taxonomy of domain-knowledge artifacts and outlines a multi-year research plan to extend SBST with diverse knowledge sources.

Abstract

Search-based Software Testing (SBST) can automatically generate test cases to search for requirements violations. Unlike manual test case development, it can generate a substantial number of test cases in a limited time. However, SBST does not possess the domain knowledge of engineers. Several techniques have been proposed to integrate engineers' domain knowledge within existing SBST frameworks. This paper will reflect on recent experimental results by highlighting bold and unexpected results. It will help re-examine SBST techniques driven by domain knowledge from a new perspective, suggesting new directions for future research.

Figures (5)

• Figure 1: SBST Driven by Domain Knowledge.
• Figure 2: Integrating Domain Knowledge within SBST by exploiting Fitness Functions.
• Figure 3: Integrating Domain Knowledge within SBST by exploiting existing test cases.
• Figure 4: Type of domain knowledge.
• Figure 5: Gantt Chart