Out-of-Distribution Detection using Synthetic Data Generation

TL;DR

This work tackles the scarcity of real out-of-distribution (OOD) data for reliable OOD detection by leveraging Large Language Models to generate high-quality synthetic OOD proxies. It provides a joint synthetic data pipeline (distinguishing near- and far-OOD) and two detector designs (repurposed InD head or end-to-end ($K+1$) classifier), evaluated across nine InD–OOD pairs in toxicity, harm, RLHF reward modeling, and selective classification tasks. Across these settings, the synthetic proxies substantially reduce false positives (FPR95) and maintain or exceed in-distribution accuracy, often matching or surpassing an ideal model trained on original OOD data. The results demonstrate the practicality of LLM-driven data synthesis for robust OOD detection in NLP and LLM development pipelines, with implications for content moderation, alignment, and data curation workflows.

Abstract

Distinguishing in- and out-of-distribution (OOD) inputs is crucial for reliable deployment of classification systems. However, OOD data is typically unavailable or difficult to collect, posing a significant challenge for accurate OOD detection. In this work, we present a method that harnesses the generative capabilities of Large Language Models (LLMs) to create high-quality synthetic OOD proxies, eliminating the dependency on any external OOD data source. We study the efficacy of our method on classical text classification tasks such as toxicity detection and sentiment classification as well as classification tasks arising in LLM development and deployment, such as training a reward model for RLHF and detecting misaligned generations. Extensive experiments on nine InD-OOD dataset pairs and various model sizes show that our approach dramatically lowers false positive rates (achieving a perfect zero in some cases) while maintaining high accuracy on in-distribution tasks, outperforming baseline methods by a significant margin.

Figures (12)

• Figure 1: A high-level illustration of synthetic data generation pipeline for OOD detection.
• Figure 2: Comparison of far- and near-OOD instances with InD samples.
• Figure 3: Risk coverage curves for Civil Comments and ToxiGen as InD-OOD pair on Llama-2 7B. Grey dashed lines mark the binary model's InD performance. The top axis represents the remaining proportion of OOD data in the coverage.
• Figure 4: UMAP mcinnes2018umap visualization of embeddings generated by a sentence transformers model (paraphrase-MiniLM-L6-v2) reimers2019sentence using CC as InD dataset. (a) Far-OOD: GSM8k and MBPP (b) Near-OOD: SST-2 and ToxiGen.
• Figure 5: Effect of LLM size on far- and near-OOD performance.