Metamorphic Relation Generation: State of the Art and Visions for Future Research
Rui Li, Huai Liu, Pak-Lok Poon, Dave Towey, Chang-Ai Sun, Zheng Zheng, Zhi Quan Zhou, Tsong Yueh Chen
TL;DR
The paper tackles the oracle problem in software testing by surveying systematic generation of metamorphic relations (MRs), the core component of metamorphic testing (MT). It catalogues seven MR-generation families—composition, AI-based techniques, MR patterns, category-choice frameworks, genetic methods, search-based approaches, and miscellaneous—in a synthesis of 75 papers published from 1998 to 2023, with 57 from 2019–2023. Key contributions include empirical insights on composite MR effectiveness, AI/LLM-driven MR prediction and generation, domain-specific MR patterns, and a set of forward-looking visions for theory, automation, and broader QA uses. The paper envisions advances in MR adequacy and diversity, automation, hybrid technique design, domain-specific MR languages, MT test-case research, and extending MR concepts beyond testing to other software quality assurance activities.
Abstract
Metamorphic testing has become one mainstream technique to address the notorious oracle problem in software testing, thanks to its great successes in revealing real-life bugs in a wide variety of software systems. Metamorphic relations, the core component of metamorphic testing, have continuously attracted research interests from both academia and industry. In the last decade, a rapidly increasing number of studies have been conducted to systematically generate metamorphic relations from various sources and for different application domains. In this article, based on the systematic review on the state of the art for metamorphic relations' generation, we summarize and highlight visions for further advancing the theory and techniques for identifying and constructing metamorphic relations, and discuss potential research trends in related areas.