Exploring Diverse Representations for Open Set Recognition

TL;DR

The paper tackles OSR by addressing open space risk with a discriminative approach that learns supplementary representations. It introduces MEDAF, a multi-expert architecture with attention-diversity regularization and an adaptive gating network to fuse expert logits, enabling diverse, complementary features for known and unknown classification. Theoretical insight shows that incorporating supplementary representations reduces open space risk via increased mutual information between features and labels, and empirical results demonstrate up to 9.5% AUROC gains on large-scale benchmarks with minimal overhead. The method is plug-and-play with existing classifiers and delivers strong performance on standard OSR tasks, near- and far-OOD detection, and large-scale ImageNet-1k benchmarks, making it practically impactful for scalable open-world recognition.

Abstract

Open set recognition (OSR) requires the model to classify samples that belong to closed sets while rejecting unknown samples during test. Currently, generative models often perform better than discriminative models in OSR, but recent studies show that generative models may be computationally infeasible or unstable on complex tasks. In this paper, we provide insights into OSR and find that learning supplementary representations can theoretically reduce the open space risk. Based on the analysis, we propose a new model, namely Multi-Expert Diverse Attention Fusion (MEDAF), that learns diverse representations in a discriminative way. MEDAF consists of multiple experts that are learned with an attention diversity regularization term to ensure the attention maps are mutually different. The logits learned by each expert are adaptively fused and used to identify the unknowns through the score function. We show that the differences in attention maps can lead to diverse representations so that the fused representations can well handle the open space. Extensive experiments are conducted on standard and OSR large-scale benchmarks. Results show that the proposed discriminative method can outperform existing generative models by up to 9.5% on AUROC and achieve new state-of-the-art performance with little computational cost. Our method can also seamlessly integrate existing classification models. Code is available at https://github.com/Vanixxz/MEDAF.

Figures (4)

• Figure 1: Illustration of the proposed MEDAF method. MEDAF consists of a multi-expert feature extractor to explore diverse representations by constraining the learned attention map of each expert to be mutually different. Then a gating network adaptively generates weights to integrate expert-independent predictions.
• Figure 2: Performance with different expert numbers on multiple datasets, with (a) recording closed-set accuracy and (b) recording AUROC.
• Figure 3: OSR performance against computational cost.
• Figure 4: Visulizations on CAMs of baseline and MEDAF.

Theorems & Definitions (8)

• Proposition 1
• Lemma 1
• Definition 1
• Definition 2
• Theorem 1
• proof
• Lemma 2
• Remark 1