Anomalous Change Point Detection Using Probabilistic Predictive Coding

TL;DR

The paper tackles the limitations of traditional change point and anomaly detection by introducing Probabilistic Predictive Coding (PPC), a deep learning framework that encodes sequential data into a latent space and predicts future latent representations with uncertainty. By training with maximum likelihood and an auxiliary reconstruction objective, PPC yields a probabilistic conformance score for each data point, enabling scalable, interpretable ACPD across diverse data modalities. The authors validate PPC through proportionality tests and four experiments—synthetic sine waves, sequential MNIST digits, and in-vivo MRSI artifacts—demonstrating strong discriminative performance and practical inference speed. They highlight linear-time complexity, applicability to high-dimensional data, and the potential for domain knowledge integration, while noting current limitations and avenues for future improvement such as contrastive learning and benchmarks.

Abstract

Change point detection (CPD) and anomaly detection (AD) are essential techniques in various fields to identify abrupt changes or abnormal data instances. However, existing methods are often constrained to univariate data, face scalability challenges with large datasets due to computational demands, and experience reduced performance with high-dimensional or intricate data, as well as hidden anomalies. Furthermore, they often lack interpretability and adaptability to domain-specific knowledge, which limits their versatility across different fields. In this work, we propose a deep learning-based CPD/AD method called Probabilistic Predictive Coding (PPC) that jointly learns to encode sequential data to low-dimensional latent space representations and to predict the subsequent data representations as well as the corresponding prediction uncertainties. The model parameters are optimized with maximum likelihood estimation by comparing these predictions with the true encodings. At the time of application, the true and predicted encodings are used to determine the probability of conformance, an interpretable and meaningful anomaly score. Furthermore, our approach has linear time complexity, scalability issues are prevented, and the method can easily be adjusted to a wide range of data types and intricate applications. We demonstrate the effectiveness and adaptability of our proposed method across synthetic time series experiments, image data, and real-world magnetic resonance spectroscopic imaging data.

Figures (11)

• Figure 1: The PPC pipeline architecture. Encoder model $E$ encodes data instances to their latent space representations, and decoder model $D$ aims to reconstruct the original data instances. Recurrent neural network model $G$ and forecasting models $F_i$ make predictions of the latent space encodings of future latent space representations given the past data instances. Maximum likelihood estimation (MLE) loss and reconstruction loss are depicted in gray.
• Figure 2: An example of ground truth and estimated probability distribution functions for three different values of $x_1$.
• Figure 3: Examples of generated signals. The top signal has no frequency deviation. The center signal has a frequency change of $1$ Hz at the change point around $t=11$ s. The bottom signal has a frequency change of $2$ Hz at the change point around $t=11$ s.
• Figure 4: Receiver Operating Characteristic and Precision-Recall curves for the application of the PPC pipeline on the sine wave frequency deviation example.
• Figure 5: Average log-likelihood (left) and probability of conformance (right) per combination of center frequencies. The frequency band margins indicate how much frequencies may fluctuate as part of the generated instantaneous frequencies $f(t)$.