ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection

TL;DR

A novel OOD detection framework that discovers ID-like outliers using CLIP from the vicinity space of the ID samples, thus helping to identify these most challenging OOD samples and elegantly exploiting the capabilities of CLIP is proposed.

Abstract

Out-of-distribution (OOD) detection methods often exploit auxiliary outliers to train model identifying OOD samples, especially discovering challenging outliers from auxiliary outliers dataset to improve OOD detection. However, they may still face limitations in effectively distinguishing between the most challenging OOD samples that are much like in-distribution (ID) data, i.e., \idlike samples. To this end, we propose a novel OOD detection framework that discovers \idlike outliers using CLIP \cite{DBLP:conf/icml/RadfordKHRGASAM21} from the vicinity space of the ID samples, thus helping to identify these most challenging OOD samples. Then a prompt learning framework is proposed that utilizes the identified \idlike outliers to further leverage the capabilities of CLIP for OOD detection. Benefiting from the powerful CLIP, we only need a small number of ID samples to learn the prompts of the model without exposing other auxiliary outlier datasets. By focusing on the most challenging \idlike OOD samples and elegantly exploiting the capabilities of CLIP, our method achieves superior few-shot learning performance on various real-world image datasets (e.g., in 4-shot OOD detection on the ImageNet-1k dataset, our method reduces the average FPR95 by 12.16\% and improves the average AUROC by 2.76\%, compared to state-of-the-art methods). Code is available at https://github.com/ycfate/ID-like.

Figures (7)

• Figure 1: Hard and easy OOD examples: hard OOD samples typically contain more features correlated to ID samples, i.e., they behave more ID-like.
• Figure 2: The standard method can only output the predicted probabilities of samples for each ID class. In contrast, our approach can automatically learn additional classes that are highly correlated but distinct from the ID classes, thereby effectively identifying challenging ID-like OOD samples. (Note that the dog-like prompt in the figure is learnable.)
• Figure 3: Overview of our method. We conduct multiple random cropping on ID sample and filter them based on their cosine similarity with established ID zero-shot prompts, thereby generating both ID and OOD data. Subsequently, prompt learning is employed to acquire prompts corresponding to the ID and ID-like OOD samples. The obtained prompts can effectively identify OOD samples in the inference stage.
• Figure 4: Density of the obtained ID and OOD score with the proposed method (left) and MCM DBLP:conf/nips/MingCGSL022 (right).
• Figure 5: The obtained representations visualization of ID-like OOD samples and ID samples under 1-shot and 4-shot settings. The representation of the obtained ID-like OOD sample is close to the ID sample.