MoDitector: Module-Directed Testing for Autonomous Driving Systems

TL;DR

MoDitector addresses the problem of root-cause attribution in ADS testing by introducing Module-Induced Critical Scenarios ($\mathcal{M}\text{ICS}$) and a three-part framework: Module-Specific Oracle, Module-Specific Feedback, and Adaptive Seed Generation. The approach computes module-level errors and a safety metric to identify whether a scenario isolates the target module’s failure, then guides fuzzing toward scenarios that amplify the targeted module’s faults while minimizing others. Empirical evaluation on Pylot-CARLA across four ADS modules and four scenarios shows MoDitector substantially outperforms baselines in discovering $\mathcal{M}\text{ICS}$, with high repair rates confirming oracle correctness. The work advances ADS debugging by providing module-level insight, enabling targeted repairs and more robust system design, while outlining extensions to multi-module causation and non-safety metrics for future work.

Abstract

Testing Autonomous Driving Systems (ADS) is crucial for ensuring their safety, reliability, and performance. Despite numerous testing methods available that can generate diverse and challenging scenarios to uncover potential vulnerabilities, these methods often treat ADS as a black-box, primarily focusing on identifying system failures like collisions or near-misses without pinpointing the specific modules responsible for these failures. Understanding the root causes of failures is essential for effective debugging and subsequent system repair. We observed that existing methods also fall short in generating diverse failures that adequately test the distinct modules of an ADS, such as perception, prediction, planning and control. To bridge this gap, we introduce MoDitector, the first root-cause-aware testing method for ADS. Unlike previous approaches, MoDitector not only generates scenarios leading to collisions but also showing which specific module triggered the failure. This method targets specific modules, creating test scenarios that highlight the weaknesses of these given modules. Specifically, our approach involves designing module-specific oracles to ascertain module failures and employs a module-directed testing strategy that includes module-specific feedback, adaptive seed selection, and mutation. This strategy guides the generation of tests that effectively provoke module-specific failures. We evaluated MoDitector across four critical ADS modules and four testing scenarios. Our approach represents a significant innovation in ADS testing by focusing on identifying and rectifying module-specific errors within the system, moving beyond conventional black-box failure detection.

Figures (2)

• Figure 1: Overview of MoDitector
• Figure 2: Cases of $\mathcal{M}\text{ICS}$s detected by MoDitector for different specific modules.

Theorems & Definitions (1)

• definition 1: $\mathcal{M}$-Induced Critical Scenario