Detecting Concept Drift in Neural Networks Using Chi-squared Goodness of Fit Testing

TL;DR

The paper addresses concept drift during neural network inference by applying the $\chi^2$ Goodness of Fit test to hidden activations across MLP, CNN, and ViT models trained on MNIST. By generating randomized activation-subset tests, it detects distributional shifts in representations without inspecting inference outputs, correlating GoF failures with drops in accuracy under simulated drift. The approach offers a practical, scalable drift monitoring signal for deployment, demonstrated across multiple architectures and drift types. This work lays groundwork for automated drift awareness in vision models and motivates further work on identifying drift causes and reducing storage needs.

Abstract

As the adoption of deep learning models has grown beyond human capacity for verification, meta-algorithms are needed to ensure reliable model inference. Concept drift detection is a field dedicated to identifying statistical shifts that is underutilized in monitoring neural networks that may encounter inference data with distributional characteristics diverging from their training data. Given the wide variety of model architectures, applications, and datasets, it is important that concept drift detection algorithms are adaptable to different inference scenarios. In this paper, we introduce an application of the $χ^2$ Goodness of Fit Hypothesis Test as a drift detection meta-algorithm applied to a multilayer perceptron, a convolutional neural network, and a transformer trained for machine vision as they are exposed to simulated drift during inference. To that end, we demonstrate how unexpected drops in accuracy due to concept drift can be detected without directly examining the inference outputs. Our approach enhances safety by ensuring models are continually evaluated for reliability across varying conditions.

Figures (5)

• Figure 1: Concept Drift Detection for Deep Learning System Diagram
• Figure 2: Drift Demonstration - left to right: baseline, inverted, AGN with $\sigma$ 0.3, 1.2, and 4.5
• Figure 3: Histogram Activations MLP Example - expected validation data compared to observed test data (left) and expected validation data compared to observed inverted test data (right)
• Figure 4: Accuracy vs. Standard Deviation for all Networks (left) and $\chi^{2}$ GoF Test Pass Rate vs. Standard Deviation for all Networks (right)
• Figure 5: NIST $\chi_A^2$ table (probability less than the critical value ==> 1 - $\alpha$)