Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-driven Verification of DNNs for Object Recognition

Clemens Otte, Yinchong Yang, Danny Benlin Oswan

TL;DR

The paper addresses the challenge of validating DNN robustness to realistic, multi-perturbation conditions in safety-critical vision tasks. It introduces a gradient-free optimization framework that searches for perturbation chains drawn from a predefined natural perturbation set, optimizing parameters to maximize IoU deterioration on clustered image subsets. The method, demonstrated on railway-track segmentation with a U-net on RailSem19 data, identifies cluster-specific weaknesses (e.g., dark scenes, wipers, urban settings) and provides actionable insights for augmented training. This approach offers a scalable, architecture-agnostic verification tool that can enhance robustness and guide data augmentation for real-world deployments.

Abstract

The paper proposes a new testing approach for Deep Neural Networks (DNN) using gradient-free optimization to find perturbation chains that successfully falsify the tested DNN, going beyond existing grid-based or combinatorial testing. Applying it to an image segmentation task of detecting railway tracks in images, we demonstrate that the approach can successfully identify weaknesses of the tested DNN regarding particular combinations of common perturbations (e.g., rain, fog, blur, noise) on specific clusters of test images.

Data-driven Verification of DNNs for Object Recognition

TL;DR

The paper addresses the challenge of validating DNN robustness to realistic, multi-perturbation conditions in safety-critical vision tasks. It introduces a gradient-free optimization framework that searches for perturbation chains drawn from a predefined natural perturbation set, optimizing parameters to maximize IoU deterioration on clustered image subsets. The method, demonstrated on railway-track segmentation with a U-net on RailSem19 data, identifies cluster-specific weaknesses (e.g., dark scenes, wipers, urban settings) and provides actionable insights for augmented training. This approach offers a scalable, architecture-agnostic verification tool that can enhance robustness and guide data augmentation for real-world deployments.

Abstract

The paper proposes a new testing approach for Deep Neural Networks (DNN) using gradient-free optimization to find perturbation chains that successfully falsify the tested DNN, going beyond existing grid-based or combinatorial testing. Applying it to an image segmentation task of detecting railway tracks in images, we demonstrate that the approach can successfully identify weaknesses of the tested DNN regarding particular combinations of common perturbations (e.g., rain, fog, blur, noise) on specific clusters of test images.
Paper Structure (10 sections, 2 equations, 3 figures)

This paper contains 10 sections, 2 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Gradient-based adversarial attacks and $\ell^p$-robustness
  4. Robustness against natural perturbations
  5. Gradient-free optimization of perturbations
  6. Bounding the perturbation strength
  7. Clustering the input images
  8. Results
  9. Conclusion
  10. Acknowledgement

Figures (3)

  • Figure 1: Falsifying a given U-net model, which was trained to segment railway tracks. (a, e) show original (unperturbed) images, (b, f) being the respective model output which is the predicted position of railway tracks. Predictions with probability $\le 0.5$ are shown in red and do not contribute to the IoU (see eq.\ref{['eq:iou']}). The approach in Sect.\ref{['sec:our_approach']} created the perturbed images (c, g) where the model output (d, h) shows maximal deterioration.
  • Figure 2: Optimizing parameter vector $\bf \theta$ to maximize model error $e$
  • Figure 3: a) Mean IoU-deterioration per cluster (descriptions with * indicate brightness perturbation being disabled to avoid trivial counterexamples). Examples (a) and (e) in Fig.\ref{['fig:example']} belong to clusters 9 and 10, respectively. b) Usage of perturbations across clusters.