Activation Matters: Test-time Activated Negative Labels for OOD Detection with Vision-Language Models

Abstract

Out-of-distribution (OOD) detection aims to identify samples that deviate from in-distribution (ID). One popular pipeline addresses this by introducing negative labels distant from ID classes and detecting OOD based on their distance to these labels. However, such labels may present poor activation on OOD samples, failing to capture the OOD characteristics. To address this, we propose \underline{T}est-time \underline{A}ctivated \underline{N}egative \underline{L}abels (TANL) by dynamically evaluating activation levels across the corpus dataset and mining candidate labels with high activation responses during the testing process. Specifically, TANL identifies high-confidence test images online and accumulates their assignment probabilities over the corpus to construct a label activation metric. Such a metric leverages historical test samples to adaptively align with the test distribution, enabling the selection of distribution-adaptive activated negative labels. By further exploring the activation information within the current testing batch, we introduce a more fine-grained, batch-adaptive variant. To fully utilize label activation knowledge, we propose an activation-aware score function that emphasizes negative labels with stronger activations, boosting performance and enhancing its robustness to the label number. Our TANL is training-free, test-efficient, and grounded in theoretical justification. Experiments on diverse backbones and wide task settings validate its effectiveness. Notably, on the large-scale ImageNet benchmark, TANL significantly reduces the FPR95 from 17.5\% to 9.8\%. Codes are available at \href{https://github.com/YBZh/OpenOOD-VLM}{YBZh/OpenOOD-VLM}.

Figures (8)

• Figure 1: Activation analyses with negative labels mined in jiang2024negative. (a) Negative labels on a specific OOD dataset exhibit a long-tailed activation score distribution. Some labels activate more strongly on the ID dataset than on OOD, potentially misleading OOD detection. (b) A small subset of negative labels strongly activates on OOD, enabling effective detection. Most labels respond similarly across ID and OOD, slightly harming detection, while some activate higher on ID, significantly degrading performance. The FPR95 results are obtained with negative labels of top activations via Eq. \ref{['equ:neglabel_score']}. These analyses use ground truth labels from ImageNet (ID) and Places (OOD) datasets.
• Figure 2: Overall framework of TANL. We dynamically explore activated negative labels from the corpus dataset in the testing process, where the activation information is measured based on the similarity between texts and the mined positive/negative images. The activation-aware score is illustrated as a simplified example of Eq. \ref{['equ:aa_score_cul']} with $M=2$ and $C=2$.
• Figure 3: Analyses on (a) number $M$ of selected negative labels, (b) selection criterion of negative labels, (c) $\alpha$ values, and (d) batch size under OpenOOD setting.
• Figure 4: Visualization of ranked corpus dataset, where candidate labels with higher activation scores are utilized.
• Figure A5: Analyses on (a) the queue length $L$, (b) threshold $\gamma$, and (c) gap value $g$ in Eq. \ref{['equ:pos_neg_img_select']} under the OpenOOD setup.