When Anomalies Depend on Context: Learning Conditional Compatibility for Anomaly Detection

TL;DR

This work tackles contextual anomaly detection, where abnormality is defined by subject–context compatibility rather than intrinsic appearance. It formalizes anomaly labels as $y = h(a,c)$ and introduces CAAD-3K, a controlled benchmark that isolates contextual violations via a cross-context split, enabling robust evaluation of generalization to unseen subject–context pairs. The proposed CoRe-CLIP framework uses a vision–language backbone with three visual branches (subject, context, global) and text refinement to learn conditional compatibility through a text-conditioned Compatibility Reasoning Module (CRM), achieving state-of-the-art results on CAAD-3K and strong generalization to MVTec-AD, VisA, and real-world OOC datasets. Empirical results, ablations, and qualitative analyses support that explicit modeling of subject–context relationships provides complementary benefits to traditional appearance-based anomaly detection. The approach offers a principled path toward robust, context-aware perception in open-world settings.

Abstract

Anomaly detection is often formulated under the assumption that abnormality is an intrinsic property of an observation, independent of context. This assumption breaks down in many real-world settings, where the same object or action may be normal or anomalous depending on latent contextual factors (e.g., running on a track versus on a highway). We revisit \emph{contextual anomaly detection}, classically defined as context-dependent abnormality, and operationalize it in the visual domain, where anomaly labels depend on subject--context compatibility rather than intrinsic appearance. To enable systematic study of this setting, we introduce CAAD-3K, a benchmark that isolates contextual anomalies by controlling subject identity while varying context. We further propose a conditional compatibility learning framework that leverages vision--language representations to model subject--context relationships under limited supervision. Our method substantially outperforms existing approaches on CAAD-3K and achieves state-of-the-art performance on MVTec-AD and VisA, demonstrating that modeling context dependence complements traditional structural anomaly detection. Our code and dataset will be publicly released.

Figures (14)

• Figure 1: Examples illustrating context-dependent normality. The same action may be normal or anomalous depending on context.
• Figure 2: Examples illustrating context-dependent normality using samples from existing OOC datasets.
• Figure 3: Overview of CoRe-CLIP. A shared CLIP radford2021learning backbone is augmented with three Context-Selective Residual (CSR) branches for subject, context, and global representations. The refined text encoder, optimized via Text Disentanglement Objectives, produces paired normal/anomalous embeddings, while the Compatibility Reasoning Module (CRM) fuses the visual streams to infer object--scene compatibility. At inference time, the same input image is passed to all three branches, and no segmentation masks or external region proposals are used.
• Figure 4: CRM branch weighting for identical actions under normal and anomalous contexts. Bar plots indicate the relative contribution of subject, context, and global representations.
• Figure 5: Loss-weight sensitivity analysis on CAAD-3K (4-shot). Each curve shows the effect of varying a single loss-term weight while keeping all others fixed at their default values. The model remains stable across a broad range of weights for both text-space objectives and CRM fusion regularizers, demonstrating that CoRe-CLIP is not overly sensitive to hyperparameter tuning.

Theorems & Definitions (3)

• Proposition 4.1: Non-identifiability under intrinsic representation collisions
• proof : Proof Sketch
• proof