Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strong Linear Baselines Strike Back: Closed-Form Linear Models as Gaussian Process Conditional Density Estimators for TSAD

Aleksandr Yugay, Hang Cui, Changhua Pei, Alexey Zaytsev

TL;DR

This work argues that closed-form linear models, specifically ordinary least squares (OLS) and reduced-rank regression (RRR) applied to lagged time-series features, provide a powerful baseline for time series anomaly detection (TSAD). The authors show that anomaly scores based on squared residuals align with finite-history Gaussian process conditional densities, offering a principled probabilistic interpretation and enabling efficient, robust detection. Across extensive univariate and multivariate benchmarks, OLS (and RR for multivariate data) consistently match or outperform state-of-the-art deep detectors while requiring dramatically fewer computational resources. The results, complemented by a theoretical link to Gaussian processes and a practical case for stronger linear baselines, advocate re-evaluating TSAD progress through principled, analytically solvable baselines and richer temporal benchmarks.

Abstract

Research in time series anomaly detection (TSAD) has largely focused on developing increasingly sophisticated, hard-to-train, and expensive-to-infer neural architectures. We revisit this paradigm and show that a simple linear autoregressive anomaly score with the closed-form solution provided by ordinary least squares (OLS) regression consistently matches or outperforms state-of-the-art deep detectors. From a theoretical perspective, we show that linear models capture a broad class of anomaly types, estimating a finite-history Gaussian process conditional density. From a practical side, across extensive univariate and multivariate benchmarks, the proposed approach achieves superior accuracy while requiring orders of magnitude fewer computational resources. Thus, future research should consistently include strong linear baselines and, more importantly, develop new benchmarks with richer temporal structures pinpointing the advantages of deep learning models.

Strong Linear Baselines Strike Back: Closed-Form Linear Models as Gaussian Process Conditional Density Estimators for TSAD

TL;DR

This work argues that closed-form linear models, specifically ordinary least squares (OLS) and reduced-rank regression (RRR) applied to lagged time-series features, provide a powerful baseline for time series anomaly detection (TSAD). The authors show that anomaly scores based on squared residuals align with finite-history Gaussian process conditional densities, offering a principled probabilistic interpretation and enabling efficient, robust detection. Across extensive univariate and multivariate benchmarks, OLS (and RR for multivariate data) consistently match or outperform state-of-the-art deep detectors while requiring dramatically fewer computational resources. The results, complemented by a theoretical link to Gaussian processes and a practical case for stronger linear baselines, advocate re-evaluating TSAD progress through principled, analytically solvable baselines and richer temporal benchmarks.

Abstract

Research in time series anomaly detection (TSAD) has largely focused on developing increasingly sophisticated, hard-to-train, and expensive-to-infer neural architectures. We revisit this paradigm and show that a simple linear autoregressive anomaly score with the closed-form solution provided by ordinary least squares (OLS) regression consistently matches or outperforms state-of-the-art deep detectors. From a theoretical perspective, we show that linear models capture a broad class of anomaly types, estimating a finite-history Gaussian process conditional density. From a practical side, across extensive univariate and multivariate benchmarks, the proposed approach achieves superior accuracy while requiring orders of magnitude fewer computational resources. Thus, future research should consistently include strong linear baselines and, more importantly, develop new benchmarks with richer temporal structures pinpointing the advantages of deep learning models.
Paper Structure (26 sections, 20 equations, 6 figures, 4 tables)

This paper contains 26 sections, 20 equations, 6 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background & Literature Review
  3. Methods
  4. Problem Setup
  5. Linear Modeling with OLS and RRR
  6. Ordinary Least Squares (OLS).
  7. Reduced-Rank Regression (RRR).
  8. Computational Complexity
  9. Linear models justification
  10. Univariate conditional density estimation
  11. Multivariate conditional density estimation
  12. Low-rank coregionalization
  13. Interpretation
  14. Results
  15. Experimental Settings
  16. ...and 11 more sections

Figures (6)

  • Figure 1: Critical Difference (CD) diagrams comparing the performance of OLS/RRR and baseline methods on the TSB-AD benchmark. Results are shown separately for univariate series (all anomalies), univariate series with point anomalies, univariate series with sequence anomalies, and multivariate series. Methods connected by a horizontal line are not significantly different according to the Nemenyi test ($p<0.05$).
  • Figure 2: Examples of different types of anomalies.
  • Figure 3: Score Machine.
  • Figure 4: The average scores of F1, F1-5, and E-F-5 under different window lengths
  • Figure 5: RRR performance across datasets for different window sizes and ranks with Min-Max scaling as a preprocessing step. Full rank (rightmost value) corresponds to OLS baseline
  • ...and 1 more figures