Known Meets Unknown: Mitigating Overconfidence in Open Set Recognition

TL;DR

Open Set Recognition (OSR) often suffers from overconfidence when unknown classes semantically resemble known classes due to inter-class overlap. The authors propose a two-component framework that combines perturbation-based uncertainty estimation with a two-stage unknown detection pipeline to explicitly mitigate this overconfidence and improve known/unknown separation. The first component perturbs model parameters to quantify predictive uncertainty, producing a robust uncertainty measure that helps distinguish known from unknown samples; the second component uses a two-stage ISDA-based alignment (with a Gaussian Mixture Model and Gaussian Naive Bayes) followed by a Decision Tree to refine the unknown boundary. Extensive experiments on SCADA, GAS, and ELECTRA show the method achieving state-of-the-art OSR performance across multiple metrics, with practical training considerations and robust hyperparameter behavior. This approach provides a principled, scalable way to address overconfidence due to feature-space overlap in open-set scenarios, with potential applicability to safety-critical recognition tasks.

Abstract

Open Set Recognition (OSR) requires models not only to accurately classify known classes but also to effectively reject unknown samples. However, when unknown samples are semantically similar to known classes, inter-class overlap in the feature space often causes models to assign unjustifiably high confidence to them, leading to misclassification as known classes -- a phenomenon known as overconfidence. This overconfidence undermines OSR by blurring the decision boundary between known and unknown classes. To address this issue, we propose a framework that explicitly mitigates overconfidence caused by inter-class overlap. The framework consists of two components: a perturbation-based uncertainty estimation module, which applies controllable parameter perturbations to generate diverse predictions and quantify predictive uncertainty, and an unknown detection module with distinct learning-based classifiers, implemented as a two-stage procedure, which leverages the estimated uncertainty to improve discrimination between known and unknown classes, thereby enhancing OSR performance. Experimental results on three public datasets show that the proposed framework achieves superior performance over existing OSR methods.

Figures (5)

• Figure 1: The overall architecture of the proposed framework. It consists of two main components: the perturbation-based uncertainty estimation module and the two-stage unknown detection module. The first component perturbs the trained classification model in the parameter space to estimate predictive uncertainty, while the second component leverages the estimated uncertainty to identify unknown samples through a two-stage decision process.
• Figure 2: Classification model architecture.
• Figure 3: Density distributions of predictive uncertainty values for known and unknown samples under different data settings.
• Figure 4: Parameter sensitivity analysis of $B$, $\lambda$, $\mu^{*}$, and $H_2$ on the GAS dataset.
• Figure 5: Accuracy variation with $H_2$ on SCADA and ELECTRA datasets.