Learning Unknowns from Unknowns: Diversified Negative Prototypes Generator for Few-Shot Open-Set Recognition

TL;DR

The DNPG model, by learning from the unknown space, generates negative prototypes that cover a broader unknown space, thereby achieving state-of-the-art performance on three standard FSOR datasets.

Abstract

Few-shot open-set recognition (FSOR) is a challenging task that requires a model to recognize known classes and identify unknown classes with limited labeled data. Existing approaches, particularly Negative-Prototype-Based methods, generate negative prototypes based solely on known class data. However, as the unknown space is infinite while the known space is limited, these methods suffer from limited representation capability. To address this limitation, we propose a novel approach, termed \textbf{D}iversified \textbf{N}egative \textbf{P}rototypes \textbf{G}enerator (DNPG), which adopts the principle of "learning unknowns from unknowns." Our method leverages the unknown space information learned from base classes to generate more representative negative prototypes for novel classes. During the pre-training phase, we learn the unknown space representation of the base classes. This representation, along with inter-class relationships, is then utilized in the meta-learning process to construct negative prototypes for novel classes. To prevent prototype collapse and ensure adaptability to varying data compositions, we introduce the Swap Alignment (SA) module. Our DNPG model, by learning from the unknown space, generates negative prototypes that cover a broader unknown space, thereby achieving state-of-the-art performance on three standard FSOR datasets.

Figures (10)

• Figure 1: Task setting for FSOR: The model undergoes a two-phase pre-training on base class data, followed by testing where it classifies novel classes (green, gray, orange boxes) and identifies unknown class samples (red box).
• Figure 2: (a) Existing SOTA FSOR methods, like ATT-G huang2022task, generate Negative Prototypes (NPs) from support samples, leading to limited diversity (e.g., "Dalmatian" NPs resemble dog-like animals). (b) Our DNPG model leverages the unknown space representation (open weights) of base classes to produce diversified NPs in the test phase.
• Figure 3: An overview of the base-class stage in our DNPG model. In the pre-training phase, the model learns an inverse representation, termed as open weight, for each base class. During the meta-learning phase, these open weights, along with the similarity relationship between base and novel classes, are utilized to generate Novel Prototypes (NPs). Furthermore, the Swap Alignment module is employed to guide the NP generation process, thereby improving their diversity.
• Figure 4: Details of the Raw Class Prototypes Calibration (RPC) and Multiple NPs Generator (MNG) modules. In the RPC module, a standard Transformer attention block is utilized to calibrate raw prototypes with base weights. Subsequently, in the MNG module, based on the similarity relationship established in the RPC module, open weights are employed to generate multiple NPs for the current episode.
• Figure 5: Visualization of prototypes and NPs for a 5-way-5-shot FSOR task on CIFAR-FS. The first row shows NPs by ATT-G, often inaccurately approximating known class prototypes (red boxes). The second row shows NPs by our DNPG, accurately representing unknown class space (gray background). Each column is a different episode.