Context-Aware Online Conformal Anomaly Detection with Prediction-Powered Data Acquisition

TL;DR

This work tackles online anomaly detection under data scarcity by introducing context-aware prediction-powered conformal online anomaly detection (C-PP-COAD). The method wraps any pre-trained anomaly score with synthetic calibration data and an active p-value mechanism that adaptively decides when to acquire real calibration data, all while ensuring decaying-memory online FDR control via the LORD procedure. It integrates context through a probabilistic rule that governs real-data queries based on proxy p-values, and uses a formal active p-value to produce valid conformal decisions even with missing data. Theoretical guarantees bound the time-average sFDR at a user-specified level and empirical results on thyroid and O-RAN datasets demonstrate improved data efficiency and robust detection power across different scoring paradigms. This approach enables reliable anomaly detection in dynamic settings with limited real calibration data, leveraging synthetic data and contextual information for practical deployment.

Abstract

Online anomaly detection is essential in fields such as cybersecurity, healthcare, and industrial monitoring, where promptly identifying deviations from expected behavior can avert critical failures or security breaches. While numerous anomaly scoring methods based on supervised or unsupervised learning have been proposed, current approaches typically rely on a continuous stream of real-world calibration data to provide assumption-free guarantees on the false discovery rate (FDR). To address the inherent challenges posed by limited real calibration data, we introduce context-aware prediction-powered conformal online anomaly detection (C-PP-COAD). Our framework strategically leverages synthetic calibration data to mitigate data scarcity, while adaptively integrating real data based on contextual cues. C-PP-COAD utilizes conformal p-values, active p-value statistics, and online FDR control mechanisms to maintain rigorous and reliable anomaly detection performance over time. Experiments conducted on both synthetic and real-world datasets demonstrate that C-PP-COAD significantly reduces dependency on real calibration data without compromising guaranteed FDR control.

Figures (8)

• Figure 1: (a) Conventional anomaly detection approaches with FDR control -- referred to as conformal online anomaly detection (COAD) -- require a continuous stream of fresh nominal data to recalibrate the scoring function used in the anomaly test. (b) The proposed approach, context-aware prediction-powered conformal online anomaly detection (C-PP-COAD) adaptively chooses between real and synthetic data using contextual information, so as to improve data efficiency while maintaining statistical reliability.
• Figure 2: An overview of C-PP-COAD
• Figure 3: Data splits for schemes considered in this work.
• Figure 4: Performance of supervised (random forest), unsupervised (clustering), and semi-supervised (one-class SVM) score functions on the Thyroid disease dataset, evaluated using a conventional fixed-threshold strategy and C-PP-COAD. The two panels show the average decaying memory sFDR and average power, respectively, as a function of time during testing. Thin lines indicate violations of the sFDR guarantee.
• Figure 5: Performance of COAD, C-PP-COAD, and context-agnostic benchmark methods on the Thyroid disease dataset. The three panels show the sFDR (\ref{['eq:FDR']}), average power \ref{['eq:power']}, and average CDAR \ref{['eq:cdar']}, respectively, as a function of time during testing. In the first two panels, thin lines indicate violations of the sFDR guarantee.

Theorems & Definitions (2)

• Proposition 1
• proof