Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization

TL;DR

The paper reframes OOD problems through a neuron-activation lens and proposes Neuron Activation Coverage (NAC), a per-neuron coverage measure built on a neuron activation state that combines output with decision influence via KL-divergence gradients. NAC comprises a PDF-based coverage function and is applied to two tasks: NAC-UE for post-hoc OOD detection and NAC-ME for evaluating model robustness to OOD shifts. Empirically, NAC-UE achieves state-of-the-art OOD detection across CIFAR-10/100 and ImageNet with minimal impact on InD accuracy, while NAC-ME shows consistent positive correlations with OOD generalization across multiple domain generalization benchmarks and backbones. The work argues that NAC offers a data-centric, layer-spanning criterion for assessing robustness and guiding model deployment in diverse, real-world settings.

Abstract

The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i.e., in-distribution (InD). In this paper, we study the OOD problem from a neuron activation view. We first formulate neuron activation states by considering both the neuron output and its influence on model decisions. Then, to characterize the relationship between neurons and OOD issues, we introduce the \textit{neuron activation coverage} (NAC) -- a simple measure for neuron behaviors under InD data. Leveraging our NAC, we show that 1) InD and OOD inputs can be largely separated based on the neuron behavior, which significantly eases the OOD detection problem and beats the 21 previous methods over three benchmarks (CIFAR-10, CIFAR-100, and ImageNet-1K). 2) a positive correlation between NAC and model generalization ability consistently holds across architectures and datasets, which enables a NAC-based criterion for evaluating model robustness. Compared to prevalent InD validation criteria, we show that NAC not only can select more robust models, but also has a stronger correlation with OOD test performance.

Figures (9)

• Figure 1: OOD detection performance on CIFAR-100 and ImageNet. AUROC scores (%) are averaged over the OOD datasets and backbones.
• Figure 2: NAC models coverage area in neuron activation space using InD training data. Upon receiving OOD data, neurons tend to behave outside the expected coverage area, thus with lower coverage scores.
• Figure 3: Illustration of our NAC-based methods. NAC is derived from the probability density function (PDF), which quantifies the coverage degree of neuron states under the InD training set $X$. Building upon NAC, we devise two approaches for tackling different OOD problems: OOD Detection (NAC-UE) and OOD Generalization (NAC-ME).
• Figure 4: OOD vs. InD neuron activation states. We employ PACS Dataset:PACSPhoto domain as InD and Sketch as OOD. All neurons stem from the layer4 of ResNet-50.
• Figure 5: Ablation studies on the neuron activation state. We visualize InD (ImageNet) and OOD (iNaturalist) distributions w.r.t. (a) neuron output, $\mathbf{z}$; (b) KL gradients of neuron output, $\partial D_{\rm KL}/\partial \mathbf{z}$; (c) our defined neuron state, $\mathbf{z} \odot \partial D_{\rm KL}/\partial \mathbf{z}$. All states are normalized via the sigmoid function.