XMutant: XAI-based Fuzzing for Deep Learning Systems

TL;DR

XMutant introduces an XAI-guided semantic fuzzing approach for DL testing, leveraging local explanations to focus mutations on high-attention input regions. The method is evaluated across text, image, and system-level driving tasks, showing increased failure exposure and faster fault discovery while maintaining high input validity and label preservation. Results indicate task-specific best configurations, with XMutant outperforming a state-of-the-art semantic-based tester and several gradient-based baselines, particularly when using attribution-based explanations. The work demonstrates that targeted, explanation-informed perturbations can improve DL generalization testing without sacrificing realism, and highlights avenues for efficiency gains and broader applicability. The findings suggest practical benefits for developers and researchers seeking scalable, valid semantic test generation guided by explainability signals.

Abstract

Semantic-based test generators are widely used to produce failure-inducing inputs for Deep Learning (DL) systems. They typically generate challenging test inputs by applying random perturbations to input semantic concepts until a failure is found or a timeout is reached. However, such randomness may hinder them from efficiently achieving their goal. This paper proposes XMutant, a technique that leverages explainable artificial intelligence (XAI) techniques to generate challenging test inputs. XMutant uses the local explanation of the input to inform the fuzz testing process and effectively guide it toward failures of the DL system under test. We evaluated different configurations of XMutant in triggering failures for different DL systems both for model-level (sentiment analysis, digit recognition) and system-level testing (advanced driving assistance). Our studies showed that XMutant enables more effective and efficient test generation by focusing on the most impactful parts of the input. XMutant generates up to 125% more failure-inducing inputs compared to an existing baseline, up to 7X faster. We also assessed the validity of these inputs, maintaining a validation rate above 89%, according to automated and human validators.

Figures (9)

• Figure 1: Example of movie review sentiment analysis case study (best viewed in color). (a) Local explanation by SmoothGrad; (b) Local explanation by LIME.
• Figure 2: XMutant steps applied to the MNIST case study (best viewed in color). (a) SVG semantic-based representation of a digit 3. Control points (i.e., semantic representation) are shown as red crosses; (b) Heatmap generated by Grad-CAM++; (c) Square windows centered on the control points; (d) Clustering on the local explanation (one cluster per control point); (e) Control point selection (red cross marker) and mutation towards the attention attractor of the cluster (green star); (f) Digit 3 resulting by mutating the candidate control point.
• Figure 3: XMutant steps for the ADAS (best viewed in color). (a) A road with 3 control points, or semantic representation (red crosses). The road sectors are indicated as Roman numerals and separated by dashed cyan lines; (b) A sequence of driving frames recorded by the vehicle's camera; (c) A sequence of heatmaps, corresponding to the driving frames, generated by SmoothGrad; (d) Weights for each road sector and associated semantic concept; (e) Control point selection (top-right) and mutation towards the right lane (dashed yellow line indicates the new centerline); (f) Road resulting from the mutation of the control point in sector III.
• Figure 4: Validity and preservation rates for sentiment analysis (left) and digit recognition (right).
• Figure 5: Semantic space visualization of generated inputs by different techniques.