Toward Cost-effective Adaptive Random Testing: An Approximate Nearest Neighbor Approach
Rubing Huang, Chenhui Cui, Junlong Lian, Dave Towey, Weifeng Sun, Haibo Chen
TL;DR
This work tackles the high computational cost of Adaptive Random Testing (ART) by introducing LSH-ART, an Approximate Nearest Neighbor (ANN) framework based on Locality-Sensitive Hashing (LSH) to replace exact NN searches in STFCS-based ART. By maintaining the diversity and fault-detection advantages of ART while significantly reducing search overhead, LSH-ART achieves lower test-case-generation costs, especially in high-dimensional input spaces, without sacrificing core effectiveness. The authors provide a framework and three concrete algorithms (LSH-STFCS, LSH-FSCS, LSH-RRT), analyze time and space complexity, and validate the approach through extensive numerical simulations and empirical studies on numerical and non-numerical domains, including configurable real-world systems. Results show that LSH-ART often matches or surpasses RT and, in many scenarios, ART variants in fault detection while delivering superior cost-effectiveness, particularly as dimensionality grows, and proves adaptable to non-numerical domains. The work also points to future directions such as exploring multiple SLSH tables, other ANN approaches, and broader constraint-handling strategies for constrained test-case generation.
Abstract
Adaptive Random Testing (ART) enhances the testing effectiveness (including fault-detection capability) of Random Testing (RT) by increasing the diversity of the random test cases throughout the input domain. Many ART algorithms have been investigated such as Fixed-Size-Candidate-Set ART (FSCS) and Restricted Random Testing (RRT), and have been widely used in many practical applications. Despite its popularity, ART suffers from the problem of high computational costs during test-case generation, especially as the number of test cases increases. Although several strategies have been proposed to enhance the ART testing efficiency, such as the forgetting strategy and the k-dimensional tree strategy, these algorithms still face some challenges, including: (1) Although these algorithms can reduce the computation time, their execution costs are still very high, especially when the number of test cases is large; and (2) To achieve low computational costs, they may sacrifice some fault-detection capability. In this paper, we propose an approach based on Approximate Nearest Neighbors (ANNs), called Locality-Sensitive Hashing ART (LSH-ART). When calculating distances among different test inputs, LSH-ART identifies the approximate (not necessarily exact) nearest neighbors for candidates in an efficient way. LSH-ART attempts to balance ART testing effectiveness and efficiency.