Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks

Charles Arnal, Felix Hensel, Mathieu Carrière, Théo Lacombe, Hiroaki Kurihara, Yuichi Ike, Frédéric Chazal

TL;DR

This work tackles covariate shift detection in neural networks by introducing MAGDiff, a representation derived from activation graphs of a pre-trained classifier. MAGDiff distances the input-specific activation graph from class-mean graphs and uses univariate KS tests on these distances (with Bonferroni correction) to detect distributional changes without retraining. Empirical results across MNIST, FMNIST, CIFAR-10, SVHN, and Imagenette show MAGDiff often outperforms the BBSD baseline that relies on confidence vectors, particularly under covariate shifts and for weaker shifts. The approach is lightweight, scalable, and integrates naturally with existing classifiers, offering a practical tool for monitoring deployed models and guiding further model improvements or data collection.

Abstract

Despite their successful application to a variety of tasks, neural networks remain limited, like other machine learning methods, by their sensitivity to shifts in the data: their performance can be severely impacted by differences in distribution between the data on which they were trained and that on which they are deployed. In this article, we propose a new family of representations, called MAGDiff, that we extract from any given neural network classifier and that allows for efficient covariate data shift detection without the need to train a new model dedicated to this task. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution, and yield powerful data- and task-adapted statistics for the two-sample tests commonly used for data set shift detection. We demonstrate this empirically by measuring the statistical powers of two-sample Kolmogorov-Smirnov (KS) tests on several different data sets and shift types, and showing that our novel representations induce significant improvements over a state-of-the-art baseline relying on the network output.

MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks

TL;DR

This work tackles covariate shift detection in neural networks by introducing MAGDiff, a representation derived from activation graphs of a pre-trained classifier. MAGDiff distances the input-specific activation graph from class-mean graphs and uses univariate KS tests on these distances (with Bonferroni correction) to detect distributional changes without retraining. Empirical results across MNIST, FMNIST, CIFAR-10, SVHN, and Imagenette show MAGDiff often outperforms the BBSD baseline that relies on confidence vectors, particularly under covariate shifts and for weaker shifts. The approach is lightweight, scalable, and integrates naturally with existing classifiers, offering a practical tool for monitoring deployed models and guiding further model improvements or data collection.

Abstract

Despite their successful application to a variety of tasks, neural networks remain limited, like other machine learning methods, by their sensitivity to shifts in the data: their performance can be severely impacted by differences in distribution between the data on which they were trained and that on which they are deployed. In this article, we propose a new family of representations, called MAGDiff, that we extract from any given neural network classifier and that allows for efficient covariate data shift detection without the need to train a new model dedicated to this task. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution, and yield powerful data- and task-adapted statistics for the two-sample tests commonly used for data set shift detection. We demonstrate this empirically by measuring the statistical powers of two-sample Kolmogorov-Smirnov (KS) tests on several different data sets and shift types, and showing that our novel representations induce significant improvements over a state-of-the-art baseline relying on the network output.
Paper Structure (36 sections, 2 theorems, 10 equations, 11 figures, 6 tables)

This paper contains 36 sections, 2 theorems, 10 equations, 11 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Shift Detection with Two-Sample Tests
  5. Neural Networks
  6. Activation Graphs
  7. Two-Sample Statistical Tests using MAGDiff
  8. The MAGDiff representations
  9. Comparison of distributions of features with multiple KS tests
  10. Differences from BBSD and motivations
  11. Experiments
  12. Experimental Settings
  13. Datasets.
  14. Architectures.
  15. Shifts.
  16. ...and 21 more sections

Key Result

Proposition 1

Let $F:X\rightarrow \mathbb{R}$ be continuous and non-constant for $X$ a separable metric space, and let $\nu \in F_*(\mathcal{P}(X))\subset \mathcal{P}(\mathbb{R})$. Then the complement $F_*^{-1}(\{\nu\})^c = \mathcal{P}(X)\backslash F_*^{-1}(\{\nu\})$ of the set $F_*^{-1}(\{\nu\})$ is a dense open

Figures (11)

  • Figure 1: Empirical distributions of $\texttt{MAGDiff}{}_1$ for the $10,000$ samples of the clean and shifted sets (MNIST, Gaussian noise, $\delta=0.5$, last dense layer). For the clean set, the distribution of the component $\texttt{MAGDiff}{}{}_1$ of MAGDiff exhibits a peak close to $0$. This corresponds to those samples whose distance to the mean activation graph of (training) samples belonging to the associated class is very small, i.e., these are samples that presumably belong to the same class as well. Note that, for the shifted set, this peak close to $0$ is substantially diminished, which indicates that the activation graph of samples affected by the shift is no longer as close to the mean activation graph of their true class.
  • Figure 2: Power and type I error of the statistical test with MAGDiff (red) and CV (green) representations w.r.t. sample size (on a log-scale) for three different shift intensities (II, IV, VI) and fixed $\delta = 0.5$ for the MNIST dataset, Gaussian noise and last layer of the network, with estimated $95\%$-confidence intervals.
  • Figure 3: Power and type I error of the test with MAGDiff (red) and CV (green) features w.r.t. the shift intensity for Gaussian noise on the MNIST dataset with sample size $100$ and $\delta = 0.25$ (left), $\delta = 0.5$ (middle), $\delta = 1.0$ (right), for the last dense layer. The estimated $95\%$-confidence intervals are displayed around the curves.
  • Figure 4: Sample images from all datasets used in the paper. From left to right: MNIST, FashionMNIST, CIFAR-10, SVHN and Imagenette.
  • Figure 5: Illustration of intensities of the shift types --- Gaussian noise (top row), Gaussian blur (middle row) and Image shift (bottom row) --- on a sample from the MNIST dataset.
  • ...and 6 more figures

Theorems & Definitions (5)

  • proof
  • Proposition 1
  • proof
  • Corollary 1
  • proof