xNose: A Test Smell Detector for C#
Partha P. Paul, Md Tonoy Akanda, M. Raihan Ullah, Dipto Mondal, Nazia S. Chowdhury, Fazle M. Tawsif
TL;DR
The paper tackles the lack of automated test smell detection in C# by mapping 16 language-agnostic smells to the C# ecosystem, developing xNose as a Roslyn-based detector usable as a Visual Studio extension or CLI, and validating it on a labeled dataset. It demonstrates high detection performance with precision $96.97\%$ and recall $96.03\%$, and conducts a large-scale prevalence study across 394 xUnit-based C# projects, revealing that about $80\%$ of test suites contain at least one smell and identifying the most frequent smells. The work contributes a practical tool and a benchmark dataset for C# test smells, enabling developers to improve test quality and enabling future cross-language smell analyses. Overall, xNose advances test quality assurance in C# by providing automated detection and empirical insights into smell distribution and co-occurrence in real-world projects.
Abstract
Test smells, similar to code smells, can negatively impact both the test code and the production code being tested. Despite extensive research on test smells in languages like Java, Scala, and Python, automated tools for detecting test smells in C# are lacking. This paper aims to bridge this gap by extending the study of test smells to C#, and developing a tool (xNose) to identify test smells in this language and analyze their distribution across projects. We identified 16 test smells from prior studies that were language-independent and had equivalent features in C# and evaluated xNose, achieving a precision score of 96.97% and a recall score of 96.03%. In addition, we conducted an empirical study to determine the prevalence of test smells in xUnit-based C# projects. This analysis sheds light on the frequency and distribution of test smells, deepening our understanding of their impact on C# projects and test suites. The development of xNose and our analysis of test smells in C# code aim to assist developers in maintaining code quality by addressing potential issues early in the development process.