Themis: Automatic and Efficient Deep Learning System Testing with Strong Fault Detection Capability

TL;DR

Themis addresses the critical need for automatic yet robust testing of Deep Learning Systems under perturbations by introducing a sensitivity-guided workflow that automatically uncovers fault-inducing data flows. It uses three components—Sensitivity Calculator, Sensitivity Convergence Coverage, and Sensitivity Maximizing Fuzzer—along with Sensitivity Sampler to estimate per-neuron convergence to a normal distribution and achieve high-probability full coverage ($95\%$). The approach empirically detects about $3.78\times$ more faults and yields $\sim14.7\times$ greater accuracy gains after retraining than four strong baselines across ten DLSs and multiple datasets, and it integrates into Mindspore’s security framework. The theoretical contribution shows activation differences $N_i(I+E)-N_i(I)$ follow a normal distribution under perturbation, enabling statistically grounded coverage, while the practical results demonstrate significant improvements in fault detection and downstream accuracy with manageable testing time. Overall, Themis advances reliable DLS deployment in safety-critical settings by providing automated, statistically sound, and scalable fault-detection capabilities without heavy manual tuning.

Abstract

Deep Learning Systems (DLSs) have been widely applied in safety-critical tasks such as autopilot. However, when a perturbed input is fed into a DLS for inference, the DLS often has incorrect outputs (i.e., faults). DLS testing techniques (e.g., DeepXplore) detect such faults by generating perturbed inputs to explore data flows that induce faults. Since a DLS often has infinitely many data flows, existing techniques require developers to manually specify a set of activation values in a DLS's neurons for exploring fault-inducing data flows. Unfortunately, recent studies show that such manual effort is tedious and can detect only a tiny proportion of fault-inducing data flows. In this paper, we present Themis, the first automatic DLS testing system, which attains strong fault detection capability by ensuring a full coverage of fault-inducing data flows at a high probability. Themis carries a new workflow for automatically and systematically revealing data flows whose internal neurons' outputs vary substantially when the inputs are slightly perturbed, as these data flows are likely fault-inducing. We evaluated Themis on ten different DLSs and found that on average the number of faults detected by Themis was 3.78X more than four notable DLS testing techniques. By retraining all evaluated DLSs with the detected faults, Themis also increased (regained) these DLSs' accuracies on average 14.7X higher than all baselines.

Figures (5)

• Figure 1: Increase in a DLS's accuracy by retraining the DLS with faults detected by Themis and the baselines. The increases in DLS's accuracy brought by Themis were on average 14.7X more than the increases in DLS's accuracy brought by baselines.
• Figure 2: Themis's architecture. Themis's components are shaded in purple. The DNN's neurons (e.g., N2) are colored in green if their MCSE value (as shown in the "Sensitivity distribution statistics") equals 0.0.
• Figure 3: Above(a): Number of faults detected by Themis and the baselines when their test coverage metric reaches 100%. Bottom(b) The mean of $\hat{ND_{i}}$ inferred by Themis for each evaluated DNN. Themis is considered accurate in inferring the mean if the variation of the mean is below 10% harding2014standard .
• Figure 4: Above (a): Average time taken by DLS testing to complete testing (achieve 100% test coverage). Bottom (b): correlation between sensitivity of neurons and the error rate.
• Figure 5: Above (a): Themis's coverage values for different sample sizes. Bottom (b): Themis's coverage values for different threshold values.