Unlocking Transfer Learning for Open-World Few-Shot Recognition

TL;DR

This work tackles Few-shot Open-Set Recognition (FSOSR), where inputs must be classified into $N$ closed-set classes or rejected as open-set. It introduces a two-stage framework, Open-set Aware Meta-Learning (Stage-1) to learn a universal FSOSR metric space with an encoder $f_ heta$ and an open-set prototype $c_ sphi$, followed by Open-set Free Transfer Learning (Stage-2) that adapts to a target task with an $(N+1)$-way classifier initialized from Stage-1. To compensate for the lack of open-set examples during transfer, it proposes two strategies: open-set sampling from a base dataset and pseudo open-set sampling from the closed set, both integrated into episodic transfer learning. The approach achieves state-of-the-art results on miniImageNet and tieredImageNet with only about 1.5% extra training cost, demonstrating that transfer learning can be effectively harnessed for FSOSR. Overall, the method shows that a task-agnostic open-set representation and practical data-generation strategies enable robust open-set detection in few-shot regimes, with strong cross-dataset performance.

Abstract

Few-Shot Open-Set Recognition (FSOSR) targets a critical real-world challenge, aiming to categorize inputs into known categories, termed closed-set classes, while identifying open-set inputs that fall outside these classes. Although transfer learning where a model is tuned to a given few-shot task has become a prominent paradigm in closed-world, we observe that it fails to expand to open-world. To unlock this challenge, we propose a two-stage method which consists of open-set aware meta-learning with open-set free transfer learning. In the open-set aware meta-learning stage, a model is trained to establish a metric space that serves as a beneficial starting point for the subsequent stage. During the open-set free transfer learning stage, the model is further adapted to a specific target task through transfer learning. Additionally, we introduce a strategy to simulate open-set examples by modifying the training dataset or generating pseudo open-set examples. The proposed method achieves state-of-the-art performance on two widely recognized benchmarks, miniImageNet and tieredImageNet, with only a 1.5\% increase in training effort. Our work demonstrates the effectiveness of transfer learning in FSOSR.

Figures (6)

• Figure 1: Difficulty of straightforward extension of the transfer learning from FSL methods IER_distillfsl_halluc to FSOSR. Compared to the pre-trained model without transfer learning (w/o TL), in open-set recognition, IER_distillfsl_halluc are less effective as much as in closed-set, or even degrade the performance.
• Figure 2: Overall training framework of OAL-OFL.(a) In Stage 1, the feature encoder and a learnable open-set prototype undergo distance-based meta-learning protonet with an additional class representing the open set. (b) In Stage 2, feature encoder and prototypes are further transfer-learned to the target task under an open-set-free condition. Open-set training examples can be alternatively drawn from the base training dataset (green) or from a subset of the closed-set categories that is randomly selected as a pseudo open set (purple).
• Figure 3: Distribution of the classification probabilities of closed-set and open-set queries to the open-set class. Closed-set and open-set queries are represented in green and red, respectively.
• Figure 4: Plots of OAL-OFL on Acc (%), AUROC (%), and loss (logarithmic scale) on iterations in Stage-2 (Best viewed in color).
• Figure 5: Plots of OAL-OFL-Lite on accuracy (%), AUROC (%), and loss (logarithmic scale) increasing iterations in Stage-2 on the 5-way 1- and 5-shot settings of the tieredImageNet.