Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Illuminating the Black Box: Real-Time Monitoring of Backdoor Unlearning in CNNs via Explainable AI

Tien Dat Hoang

TL;DR

The paper tackles the challenge of removing backdoors from CNNs in a transparent, verifiable way. It introduces a real-time monitoring framework that embeds Grad-CAM into the unlearning loop, and a Trigger Attention Ratio (TAR) metric to quantify attention shifts from triggers to legitimate object features. A balanced unlearning strategy combines gradient ascent on poisoned samples, Elastic Weight Consolidation to mitigate forgetting, and a recovery phase to restore clean accuracy, demonstrated on CIFAR-10 with BadNets where ASR drops from 96.51% to 5.52% while preserving 82.06% accuracy. This approach provides observable, interpretable evidence of backdoor removal, offering practical benefits for security practitioners and paving the way for broader adoption of explainable AI in defense workflows.

Abstract

Backdoor attacks pose severe security threats to deep neural networks by embedding malicious triggers that force misclassification. While machine unlearning techniques can remove backdoor behaviors, current methods lack transparency and real-time interpretability. This paper introduces a novel framework that integrates Gradient-weighted Class Activation Mapping (Grad-CAM) into the unlearning process to provide real-time monitoring and explainability. We propose the Trigger Attention Ratio (TAR) metric to quantitatively measure the model's attention shift from trigger patterns to legitimate object features. Our balanced unlearning strategy combines gradient ascent on backdoor samples, Elastic Weight Consolidation (EWC) for catastrophic forgetting prevention, and a recovery phase for clean accuracy restoration. Experiments on CIFAR-10 with BadNets attacks demonstrate that our approach reduces Attack Success Rate (ASR) from 96.51% to 5.52% while retaining 99.48% of clean accuracy (82.06%), achieving a 94.28% ASR reduction. The integration of explainable AI enables transparent, observable, and verifiable backdoor removal.

Illuminating the Black Box: Real-Time Monitoring of Backdoor Unlearning in CNNs via Explainable AI

TL;DR

The paper tackles the challenge of removing backdoors from CNNs in a transparent, verifiable way. It introduces a real-time monitoring framework that embeds Grad-CAM into the unlearning loop, and a Trigger Attention Ratio (TAR) metric to quantify attention shifts from triggers to legitimate object features. A balanced unlearning strategy combines gradient ascent on poisoned samples, Elastic Weight Consolidation to mitigate forgetting, and a recovery phase to restore clean accuracy, demonstrated on CIFAR-10 with BadNets where ASR drops from 96.51% to 5.52% while preserving 82.06% accuracy. This approach provides observable, interpretable evidence of backdoor removal, offering practical benefits for security practitioners and paving the way for broader adoption of explainable AI in defense workflows.

Abstract

Backdoor attacks pose severe security threats to deep neural networks by embedding malicious triggers that force misclassification. While machine unlearning techniques can remove backdoor behaviors, current methods lack transparency and real-time interpretability. This paper introduces a novel framework that integrates Gradient-weighted Class Activation Mapping (Grad-CAM) into the unlearning process to provide real-time monitoring and explainability. We propose the Trigger Attention Ratio (TAR) metric to quantitatively measure the model's attention shift from trigger patterns to legitimate object features. Our balanced unlearning strategy combines gradient ascent on backdoor samples, Elastic Weight Consolidation (EWC) for catastrophic forgetting prevention, and a recovery phase for clean accuracy restoration. Experiments on CIFAR-10 with BadNets attacks demonstrate that our approach reduces Attack Success Rate (ASR) from 96.51% to 5.52% while retaining 99.48% of clean accuracy (82.06%), achieving a 94.28% ASR reduction. The integration of explainable AI enables transparent, observable, and verifiable backdoor removal.

Paper Structure

This paper contains 25 sections, 8 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Motivation
  3. Research Gap
  4. Contributions
  5. Related Work
  6. Backdoor Attacks and Defenses
  7. Machine Unlearning
  8. Explainable AI in Security
  9. Methodology
  10. Threat Model
  11. Model Architecture
  12. Proposed Framework
  13. Phase 1: Unlearning with Combined Loss
  14. Phase 2: Recovery Phase
  15. Phase 3: Real-Time Monitoring
  16. ...and 10 more sections

Figures (4)

  • Figure 1: Backdoor trigger examples: clean (left) vs poisoned (right) with 4$\times$4 white trigger at bottom-right.
  • Figure 2: BadNet Grad-CAM: intense activation (red) on trigger region.
  • Figure 3: Unlearned Model Grad-CAM: attention distributed across object features.
  • Figure 4: TAR distribution: BadNet (red) vs Unlearned (green). Mean decreased from 1.876 to 0.604 (67.8% reduction).