Retrieval Augmented Anomaly Detection (RAAD): Nimble Model Adjustment Without Retraining

TL;DR

The paper tackles false positive reduction in real-time anomaly detection without retraining. It introduces Retrieval Augmented Anomaly Detection (RAAD), a post-processing method that stores human-corrected mispredictions in a vector store and uses embedding similarity to adjust upcoming predictions. It provides two adjustment mechanisms—probability-bounded and loss-bounded—validated across image, text, and graph modalities with multiple model architectures, showing substantial FP reductions. The work discusses embedding-space requirements, provides measurements and criteria for suitability, and outlines limitations and future directions, including metric learning and richer feedback. This approach enables real-time, low-cost improvement of anomaly detectors in production settings by leveraging human-in-the-loop feedback without retraining.

Abstract

We propose a novel mechanism for real-time (human-in-the-loop) feedback focused on false positive reduction to enhance anomaly detection models. It was designed for the lightweight deployment of a behavioral network anomaly detection model. This methodology is easily integrable to similar domains that require a premium on throughput while maintaining high precision. In this paper, we introduce Retrieval Augmented Anomaly Detection, a novel method taking inspiration from Retrieval Augmented Generation. Human annotated examples are sent to a vector store, which can modify model outputs on the very next processed batch for model inference. To demonstrate the generalization of this technique, we benchmarked several different model architectures and multiple data modalities, including images, text, and graph-based data.

Figures (3)

• Figure 1: RAAD Architecture
• Figure 2: Jaccard Index of several models and embeddings
• Figure 3: MNIST Example (Before/After RAAD)