Know Yourself Better: Diverse Object-Related Features Improve Open Set Recognition

TL;DR

This work investigates Open Set Recognition (OSR) through the lens of feature diversity. It hypothesizes that learning a wider set of object-related features enhances the model's ability to detect novel classes and proposes an ensemble approach that combines supervised contrastive models trained with different temperatures, followed by aggregation of their representations for outlier detection. Empirical results on standard OSR benchmarks show competitive or superior AUROC and OSCR performance, with notable gains on harder datasets like TinyImageNet. The study provides practical guidance on leveraging temperature-driven representation diversity and aggregation to boost OSR in real-world deployments.

Abstract

Open set recognition (OSR) is a critical aspect of machine learning, addressing the challenge of detecting novel classes during inference. Within the realm of deep learning, neural classifiers trained on a closed set of data typically struggle to identify novel classes, leading to erroneous predictions. To address this issue, various heuristic methods have been proposed, allowing models to express uncertainty by stating "I don't know." However, a gap in the literature remains, as there has been limited exploration of the underlying mechanisms of these methods. In this paper, we conduct an analysis of open set recognition methods, focusing on the aspect of feature diversity. Our research reveals a significant correlation between learning diverse discriminative features and enhancing OSR performance. Building on this insight, we propose a novel OSR approach that leverages the advantages of feature diversity. The efficacy of our method is substantiated through rigorous evaluation on a standard OSR testbench, demonstrating a substantial improvement over state-of-the-art methods.

Figures (6)

• Figure 1: Illustration of the intrinsic link between feature diversity and OSR performance. Consider the following scenario: Inliers, cats and dogs in this example, can be accurately classified by leveraging a discriminative feature such as feather patterns. However, when faced with outliers, leopards, polar bears, and leopard-patterned jackets, relying solely on the feather textures becomes problematic, especially when these outliers exhibit high similarity in these particular feature textures. In such a case, additional features, such as the shapes of the ears and tails, need to be learned to enable the model to discern and handle a wider range of outliers effectively.
• Figure 1: Settings for the two groups of controlled experiments in Section \ref{['sec-toy-example']}. Blue circles and red rectangles are inliers in E1 and red circles are additionally introduced in E2. The inlier classification accuracy for E1 and E2 is $100\%$ and $95.33\%$ respectively. And the outliers are blue rectangles for both E1 and E2.
• Figure 2: Examples from the synthetic dataset in the controlled experiments, which are (from left to right, up to down) blue circle, red rectangle, red circle, and blue rectangle. All backgrounds are set to be black.
• Figure 3: Examples from the synthetic dataset in the controlled experiments. The circles and rectangles are not filled to evaluate if the model can recognize shapes.
• Figure 4: Left: Plots of $\frac{\partial \mathcal{L}_\mathit{SupCon}}{\partial s_\mathit{ip}}$ values with respect to $s_\mathit{ip}$ under different $\tau$ vlues. Right: Plots of $\frac{\partial \mathcal{L}_\mathit{SupCon}}{\partial s_\mathit{in}}$ values with respect to $s_\mathit{in}$ with different $\tau$'s (the curves of $\tau=0.01$ and $\tau=0.005$ are overlapped).