Don't Miss Out on Novelty: Importance of Novel Features for Deep Anomaly Detection

TL;DR

This paper addresses the limitation of deep anomaly detection methods that rely solely on familiarity by introducing a joint model that accounts for both familiar and novel features. It combines a Familiar Feature Based Anomaly Score (FFS) with an Explanation Based Novelty Score (ENS) derived from faithful $\operatorname{B-cos}$ explanations, and adapts novelty computation across anomaly types by selecting different network layers. The approach achieves state-of-the-art performance on eight benchmarks spanning semantic near/far and sensory AD, reducing false negatives by up to 40% and eliminating the need for expensive background models. It also provides interpretable, pixel-level explanations for sensory anomalies, and demonstrates robustness to the choice of outlier modeling. The work advances practical AD by integrating explainability with novelty detection, with implications for broader tasks like novel class discovery and out-of-distribution detection.

Abstract

Anomaly Detection (AD) is a critical task that involves identifying observations that do not conform to a learned model of normality. Prior work in deep AD is predominantly based on a familiarity hypothesis, where familiar features serve as the reference in a pre-trained embedding space. While this strategy has proven highly successful, it turns out that it causes consistent false negatives when anomalies consist of truly novel features that are not well captured by the pre-trained encoding. We propose a novel approach to AD using explainability to capture such novel features as unexplained observations in the input space. We achieve strong performance across a wide range of anomaly benchmarks by combining familiarity and novelty in a hybrid approach. Our approach establishes a new state-of-the-art across multiple benchmarks, handling diverse anomaly types while eliminating the need for expensive background models and dense matching. In particular, we show that by taking account of novel features, we reduce false negative anomalies by up to 40% on challenging benchmarks compared to the state-of-the-art. Our method gives visually inspectable explanations for pixel-level anomalies.

Figures (10)

• Figure 1: Example anomaly detection problem on CIFAR-100: rocket$=$normal; CIFAR-10: all classes$=$anomaly. Familiarity only causes false positives as rockets are close to planes in the embedding space and hence have a too-low familiarity-based anomaly score. (Right) our method corrects more false positives by taking novel features into account.
• Figure 2: Illustration of predominant anomaly types considered in prior work.
• Figure 3: Feature learning based AD methods succeed by detecting the presence and absence of familiar features in the test sample. Familiar features are the features the encoder learns to discriminate the normal samples from the outliers. The detection method fails for samples with novel features that the encoder is not trained to represent in the feature space.
• Figure 4: Figure shows the proposed pipeline. The top portion computes Explanation-based Novelty Score (ENS) and the bottom branch computes Familiar Feature based anomaly Score (FFS). The final score, novelty accounted anomaly score is a combination of both.
• Figure 5: Comparing the false positives across different datasets with and without accounting for novelty. Y-axis: left shows the rate of FP, and the right side shows the % reduction in FP.