Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Supporting Safety Analysis of Image-processing DNNs through Clustering-based Approaches

Mohammed Oualid Attaoui, Hazem Fahmy, Fabrizio Pastore, Lionel Briand

TL;DR

The paper investigates safety-aware root-cause analysis for image-processing DNNs by exhaustively evaluating 99 pipelines that combine feature extraction, dimensionality reduction, and clustering to produce RCCs. It benchmarks HUDD and SAFE variants using transfer-learning, autoencoders, and heatmaps, with PCA and UMAP, and clustering by K-means, DBSCAN, or HDBSCAN, across six automotive DNNs under injected and pre-existing failure scenarios. The study finds that a non-fine-tuned transfer-learning approach (notably VGG-16) combined with UMAP for dimensionality reduction and DBSCAN for clustering delivers RCCs with high purity (~94.3%) and high coverage (~96.7%), robust even for rare failures. These results offer practical guidance for safety analysis workflows, enabling engineers to identify and address failure root causes more efficiently while potentially reducing inspection costs.

Abstract

The adoption of deep neural networks (DNNs) in safety-critical contexts is often prevented by the lack of effective means to explain their results, especially when they are erroneous. In our previous work, we proposed a white-box approach (HUDD) and a black-box approach (SAFE) to automatically characterize DNN failures. They both identify clusters of similar images from a potentially large set of images leading to DNN failures. However, the analysis pipelines for HUDD and SAFE were instantiated in specific ways according to common practices, deferring the analysis of other pipelines to future work. In this paper, we report on an empirical evaluation of 99 different pipelines for root cause analysis of DNN failures. They combine transfer learning, autoencoders, heatmaps of neuron relevance, dimensionality reduction techniques, and different clustering algorithms. Our results show that the best pipeline combines transfer learning, DBSCAN, and UMAP. It leads to clusters almost exclusively capturing images of the same failure scenario, thus facilitating root cause analysis. Further, it generates distinct clusters for each root cause of failure, thus enabling engineers to detect all the unsafe scenarios. Interestingly, these results hold even for failure scenarios that are only observed in a small percentage of the failing images.

Supporting Safety Analysis of Image-processing DNNs through Clustering-based Approaches

TL;DR

The paper investigates safety-aware root-cause analysis for image-processing DNNs by exhaustively evaluating 99 pipelines that combine feature extraction, dimensionality reduction, and clustering to produce RCCs. It benchmarks HUDD and SAFE variants using transfer-learning, autoencoders, and heatmaps, with PCA and UMAP, and clustering by K-means, DBSCAN, or HDBSCAN, across six automotive DNNs under injected and pre-existing failure scenarios. The study finds that a non-fine-tuned transfer-learning approach (notably VGG-16) combined with UMAP for dimensionality reduction and DBSCAN for clustering delivers RCCs with high purity (~94.3%) and high coverage (~96.7%), robust even for rare failures. These results offer practical guidance for safety analysis workflows, enabling engineers to identify and address failure root causes more efficiently while potentially reducing inspection costs.

Abstract

The adoption of deep neural networks (DNNs) in safety-critical contexts is often prevented by the lack of effective means to explain their results, especially when they are erroneous. In our previous work, we proposed a white-box approach (HUDD) and a black-box approach (SAFE) to automatically characterize DNN failures. They both identify clusters of similar images from a potentially large set of images leading to DNN failures. However, the analysis pipelines for HUDD and SAFE were instantiated in specific ways according to common practices, deferring the analysis of other pipelines to future work. In this paper, we report on an empirical evaluation of 99 different pipelines for root cause analysis of DNN failures. They combine transfer learning, autoencoders, heatmaps of neuron relevance, dimensionality reduction techniques, and different clustering algorithms. Our results show that the best pipeline combines transfer learning, DBSCAN, and UMAP. It leads to clusters almost exclusively capturing images of the same failure scenario, thus facilitating root cause analysis. Further, it generates distinct clusters for each root cause of failure, thus enabling engineers to detect all the unsafe scenarios. Interestingly, these results hold even for failure scenarios that are only observed in a small percentage of the failing images.
Paper Structure (55 sections, 9 equations, 15 figures, 12 tables)

This paper contains 55 sections, 9 equations, 15 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Clustering
  4. Heatmap-based DNN Explanations
  5. Heatmap-based Unsupervised Debugging of DNNs (HUDD)
  6. Heatmap-based Clustering in HUDD
  7. Safety Analysis based on Feature Extraction (SAFE)
  8. Transfer Learning and Feature Extraction
  9. Dimensionality Reduction
  10. Autoencoders
  11. The proposed pipelines
  12. Feature Extraction
  13. Feature Extraction based on Transfer Learning
  14. Fine-tuning
  15. Feature Extraction based on Autoencoders
  16. ...and 40 more sections

Figures (15)

  • Figure 1: Layer-Wise Relevance Propagation.
  • Figure 2: An example image of HPD subject (on the left) and applied LRP (on the right) showing that the mouth had a large influence on the DNN behavior.
  • Figure 3: Overview of HUDD.
  • Figure 4: Generation of root cause clusters with SAFE
  • Figure 5: Autoencoder Architecture
  • ...and 10 more figures