Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection

Pei Wang, Keqing He, Yutao Mou, Xiaoshuai Song, Yanan Wu, Jingang Wang, Yunsen Xian, Xunliang Cai, Weiran Xu

TL;DR

This work tackles few-shot OOD detection by leveraging limited labeled IND data alongside large unlabeled mixed data. It introduces Adaptive Prototypical Pseudo-Labeling (APP), which combines a prototypical OOD detection framework (ProtoOOD) with adaptive pseudo-labeling to learn discriminative IND representations and reliably label unlabeled data. The method achieves strong improvements over baselines on Banking and StackOverflow under various few-shot settings, with ProtoOOD providing robust IND/OOD separation and APP enabling effective self-training through instance-prototype margins. The results suggest that prototype-centric learning and adaptive pseudo-labeling offer practical, scalable gains for real-world dialog systems facing scarce labeled data. The approach has broad implications for deploying OOD detectors in low-resource environments and can be extended with additional SSL techniques to close the gap to full-data performance.

Abstract

Detecting out-of-domain (OOD) intents from user queries is essential for a task-oriented dialogue system. Previous OOD detection studies generally work on the assumption that plenty of labeled IND intents exist. In this paper, we focus on a more practical few-shot OOD setting where there are only a few labeled IND data and massive unlabeled mixed data that may belong to IND or OOD. The new scenario carries two key challenges: learning discriminative representations using limited IND data and leveraging unlabeled mixed data. Therefore, we propose an adaptive prototypical pseudo-labeling (APP) method for few-shot OOD detection, including a prototypical OOD detection framework (ProtoOOD) to facilitate low-resource OOD detection using limited IND data, and an adaptive pseudo-labeling method to produce high-quality pseudo OOD\&IND labels. Extensive experiments and analysis demonstrate the effectiveness of our method for few-shot OOD detection.

APP: Adaptive Prototypical Pseudo-Labeling for Few-shot OOD Detection

TL;DR

This work tackles few-shot OOD detection by leveraging limited labeled IND data alongside large unlabeled mixed data. It introduces Adaptive Prototypical Pseudo-Labeling (APP), which combines a prototypical OOD detection framework (ProtoOOD) with adaptive pseudo-labeling to learn discriminative IND representations and reliably label unlabeled data. The method achieves strong improvements over baselines on Banking and StackOverflow under various few-shot settings, with ProtoOOD providing robust IND/OOD separation and APP enabling effective self-training through instance-prototype margins. The results suggest that prototype-centric learning and adaptive pseudo-labeling offer practical, scalable gains for real-world dialog systems facing scarce labeled data. The approach has broad implications for deploying OOD detectors in low-resource environments and can be extended with additional SSL techniques to close the gap to full-data performance.

Abstract

Detecting out-of-domain (OOD) intents from user queries is essential for a task-oriented dialogue system. Previous OOD detection studies generally work on the assumption that plenty of labeled IND intents exist. In this paper, we focus on a more practical few-shot OOD setting where there are only a few labeled IND data and massive unlabeled mixed data that may belong to IND or OOD. The new scenario carries two key challenges: learning discriminative representations using limited IND data and leveraging unlabeled mixed data. Therefore, we propose an adaptive prototypical pseudo-labeling (APP) method for few-shot OOD detection, including a prototypical OOD detection framework (ProtoOOD) to facilitate low-resource OOD detection using limited IND data, and an adaptive pseudo-labeling method to produce high-quality pseudo OOD\&IND labels. Extensive experiments and analysis demonstrate the effectiveness of our method for few-shot OOD detection.
Paper Structure (22 sections, 4 equations, 8 figures, 9 tables, 1 algorithm)

This paper contains 22 sections, 4 equations, 8 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Formulation
  4. Adaptive Prototypical Pseudo-Labeling
  5. Experiments
  6. Datasets
  7. Baselines
  8. Main Results
  9. Qualitative Analysis
  10. Effect of Prototypical OOD Detection
  11. Effect of Adaptive Pseudo-Labeling
  12. Hyper-parameter Analysis
  13. Effect of Different Ratios of OOD Data
  14. Few-shot OOD detection of ChatGPT
  15. Related Work
  16. ...and 7 more sections

Figures (8)

  • Figure 1: The overall architecture of our adaptive prototypical pseudo-labeling (APP) for few-shot OOD detection.
  • Figure 2: Score distribution curves of IND and OOD data using different scoring functions.
  • Figure 3: Visualization of IND and OOD intents using different IND pre-training losses.
  • Figure 4: Change of score distribution curves of IND and OOD data during adaptive pseudo-labeling.
  • Figure 5: Visualization of IND and OOD intents during adaptive pseudo-labeling.
  • ...and 3 more figures