Shaping Parameter Contribution Patterns for Out-of-Distribution Detection

TL;DR

A simple yet effective method called Shaping Parameter Contribution Patterns (SPCP), which enhances OOD detection robustness by encouraging the classifier to learn boundary-oriented dense contribution patterns, while preserving in-distribution (ID) performance.

Abstract

Out-of-distribution (OOD) detection is a well-known challenge due to deep models often producing overconfident. In this paper, we reveal a key insight that trained classifiers tend to rely on sparse parameter contribution patterns, meaning that only a few dominant parameters drive predictions. This brittleness can be exploited by OOD inputs that anomalously trigger these parameters, resulting in overconfident predictions. To address this issue, we propose a simple yet effective method called Shaping Parameter Contribution Patterns (SPCP), which enhances OOD detection robustness by encouraging the classifier to learn boundary-oriented dense contribution patterns. Specifically, SPCP operates during training by rectifying excessively high parameter contributions based on a dynamically estimated threshold. This mechanism promotes the classifier to rely on a broader set of parameters for decision-making, thereby reducing the risk of overconfident predictions caused by anomalously triggered parameters, while preserving in-distribution (ID) performance. Extensive experiments under various OOD detection setups verify the effectiveness of SPCP.

Figures (5)

• Figure 1: An illustrative example of parameter contribution patterns in a ResNet-18 classifier trained on CIFAR-10 using cross-entropy loss. Units are sorted in the same order. This example focuses on the ‘airplane’ class, showcasing the contributions of each unit in the classifier weights to the model outputs for both ID and OOD samples. The parameter contributions are defined with reference to Eq. \ref{['eq:contribution']}. Since model outputs are directly determined by parameter contributions, OOD input that anomalously triggers dominant parameters can be confidently yet incorrectly classified as ID categories and hurt OOD detection.
• Figure 2: Comparison of the classifier's parameter contribution patterns before and after applying SPCP, with the average contribution matrix on the ID test set sorted for clarity.
• Figure 3: Comparison of normalized OOD score distributions before and after applying SPCP.
• Figure 4: Ablation studies on the hyperparameters: (a) effect of varying the percentile $\rho$ used for threshold estimation; (b) impact of the EMA smoothing factor $\beta$; (c) influence of initial threshold $\lambda_0$. The OOD results are averaged over both near- and far-OOD groups on the CIFAR-100 benchmark.
• Figure 5: Visualization of the classifier’s parameter contribution patterns, with the average contribution matrix on the ID test set sorted for improved clarity.