DeepCover: Advancing RNN Test Coverage and Online Error Prediction using State Machine Extraction

TL;DR

DeepCover addresses the explainability gap in recurrent neural networks by extracting a state machine that abstracts internal state trajectories. It introduces four SM quality metrics (Purity, Richness, Goodness, Scale) and six SM-based coverage criteria to quantify test suite effectiveness, plus a tree-based online error-prediction model that uses SM-derived features to forecast run-time errors with AUCs above 0.8. The approach is validated on MNIST and Mini Speech Commands using GRU, LSTM, and S-RNN modules, showing superior SM quality and robust error-prediction performance compared with prior DeepStellar methods. The findings demonstrate that SM-based explainability supports targeted testing and runtime monitoring, with practical implications for improving dependability in sequential AI systems. Limitations include focus on recurrent architectures and the need for labeled errors for training the predictor, with future work extending to transformer-based sequential models.

Abstract

Recurrent neural networks (RNNs) have emerged as powerful tools for processing sequential data in various fields, including natural language processing and speech recognition. However, the lack of explainability in RNN models has limited their interpretability, posing challenges in understanding their internal workings. To address this issue, this paper proposes a methodology for extracting a state machine (SM) from an RNN-based model to provide insights into its internal function. The proposed SM extraction algorithm was assessed using four newly proposed metrics: Purity, Richness, Goodness, and Scale. The proposed methodology along with its assessment metrics contribute to increasing explainability in RNN models by providing a clear representation of their internal decision making process through the extracted SM. In addition to improving the explainability of RNNs, the extracted SM can be used to advance testing and and monitoring of the primary RNN-based model. To enhance RNN testing, we introduce six model coverage criteria based on the extracted SM, serving as metrics for evaluating the effectiveness of test suites designed to analyze the primary model. We also propose a tree-based model to predict the error probability of the primary model for each input based on the extracted SM. We evaluated our proposed online error prediction approach using the MNIST dataset and Mini Speech Commands dataset, achieving an area under the curve (AUC) exceeding 80\% for the receiver operating characteristic (ROC) chart.

Figures (9)

• Figure 1: DeepCover Framework
• Figure 2: General Function Schema of an RNN Module
• Figure 3: Internal Architecture of an S-RNN
• Figure 4: Internal Architecture of an LSTM Module
• Figure 5: Internal Architecture of a GRU Module