All Beings Are Equal in Open Set Recognition

TL;DR

This work addresses open-set recognition by arguing that modeling unknowns as a single K+1 class distorts their distributions and creates imbalances. It introduces Target-Aware Universum (TAU) to generate K pseudo-unknowns aligned with targeted knowns via Targeted Mixup, and a Dual Contrastive (DC) loss that treats all instances (known and TAU) as positives against negatives, promoting balanced, boundary-focused representations. The proposed Dual Contrastive Learning with Target-Aware Universum (DCTAU) achieves state-of-the-art results on multiple OSR benchmarks and exhibits strong performance in out-of-distribution detection, supported by theoretical analysis of fair contrast and hard negative mining. The approach provides a practical, scalable alternative to infinite unknowns, with ablations confirming the importance of TAU and the DC loss in achieving equal treatment of known and unknown classes. Overall, DCTAU advances boundary-aware learning in OSR with a principled K+K framework that improves both detection of unknowns and accurate classification of knowns.

Abstract

In open-set recognition (OSR), a promising strategy is exploiting pseudo-unknown data outside given $K$ known classes as an additional $K$+$1$-th class to explicitly model potential open space. However, treating unknown classes without distinction is unequal for them relative to known classes due to the category-agnostic and scale-agnostic of the unknowns. This inevitably not only disrupts the inherent distributions of unknown classes but also incurs both class-wise and instance-wise imbalances between known and unknown classes. Ideally, the OSR problem should model the whole class space as $K$+$\infty$, but enumerating all unknowns is impractical. Since the core of OSR is to effectively model the boundaries of known classes, this means just focusing on the unknowns nearing the boundaries of targeted known classes seems sufficient. Thus, as a compromise, we convert the open classes from infinite to $K$, with a novel concept Target-Aware Universum (TAU) and propose a simple yet effective framework Dual Contrastive Learning with Target-Aware Universum (DCTAU). In details, guided by the targeted known classes, TAU automatically expands the unknown classes from the previous $1$ to $K$, effectively alleviating the distribution disruption and the imbalance issues mentioned above. Then, a novel Dual Contrastive (DC) loss is designed, where all instances irrespective of known or TAU are considered as positives to contrast with their respective negatives. Experimental results indicate DCTAU sets a new state-of-the-art.

Figures (6)

• Figure 1: (a) The $K$+$1$ stereotype (top) inevitably disrupts the inherent distributions of unknowns and incurs a bigger unknown class overwhelming other known classes. The $K$+$K$ strategy introduced (below) can alleviate the issues existing in $K$+$1$; (b) An illustrated experiment on partial data of CIFAR10 indicates $K$+$K$ can be as a compromise. $K$+$K$ (red) outperforms $K$+$1$ (yellow) by a wide margin and shows comparable performance to $K$+$2K$ (blue), while requiring less time cost.
• Figure 2: Two components of the proposed framework Dual Contrastive Learning with Target-Aware Universum (DCTAU). (a) An illustration about how Target-Aware Universum(TAU) is generated; (b) The Dual Contrastive Loss is defined between the $i$-th targeted known class and other known classes & the $i$-th TAU class (top) and between the $i$-th TAU class and other TAU classes & the $i$-th targeted known class (below).
• Figure 3: The generated images from mixing digital "1" and "2" may belong to digital "2", "4" and "8" images of known classes(top). TAU can highlight the targeted known digital "1" and avoid the ambiguous samples (below).
• Figure 4: (a) AUROC and OSCR of DCTAU and DCTAU(w/o DC) with varying epochs. The experiments are conducted on CIFAR10; (b) AUROC of DCTAU with varying augmentation techniques. The experiments are conducted on CIFAR10 and TinyImageNet.
• Figure 5: Ablation of Hyper-parameters $\epsilon$.