Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection

Xinyi Wang, Lang Tong

TL;DR

This work introduces the Innovations Autoencoder (IAE), a causal convolutional framework that learns an innovations sequence $(\nu_t)$—a time-series signature that is independent of the past and, under ideal conditions, uniformly distributed. By combining a causal CNN encoder with discriminators guided by a Wasserstein GAN objective and a reconstruction loss, the IAE produces a finite-dimensional approximation to the innovations representation with a proven finite-block convergence property. The authors leverage this transformation to tackle one-class anomalous sequence detection with unknown anomaly and anomaly-free models by reducing detection to a uniformity test on the innovations, enabling Chernoff-consistent decisions. Empirical results on field data from power systems (BESS and UTK) and synthetic datasets demonstrate that IAE achieves superior or competitive anomaly detection performance (AUROC values up to 0.96) and robust independence properties compared with various baselines such as OCSVM, ANICA, and f-AnoGAN. The approach provides a practical, data-driven pathway to Wiener-style signature extraction for real-time monitoring and anomaly detection in complex, unknown-temporal-structure processes.

Abstract

An innovations sequence of a time series is a sequence of independent and identically distributed random variables with which the original time series has a causal representation. The innovation at a time is statistically independent of the history of the time series. As such, it represents the new information contained at present but not in the past. Because of its simple probability structure, an innovations sequence is the most efficient signature of the original. Unlike the principle or independent component analysis representations, an innovations sequence preserves not only the complete statistical properties but also the temporal order of the original time series. An long-standing open problem is to find a computationally tractable way to extract an innovations sequence of non-Gaussian processes. This paper presents a deep learning approach, referred to as Innovations Autoencoder (IAE), that extracts innovations sequences using a causal convolutional neural network. An application of IAE to the one-class anomalous sequence detection problem with unknown anomaly and anomaly-free models is also presented.

Innovations Autoencoder and its Application in One-class Anomalous Sequence Detection

TL;DR

This work introduces the Innovations Autoencoder (IAE), a causal convolutional framework that learns an innovations sequence —a time-series signature that is independent of the past and, under ideal conditions, uniformly distributed. By combining a causal CNN encoder with discriminators guided by a Wasserstein GAN objective and a reconstruction loss, the IAE produces a finite-dimensional approximation to the innovations representation with a proven finite-block convergence property. The authors leverage this transformation to tackle one-class anomalous sequence detection with unknown anomaly and anomaly-free models by reducing detection to a uniformity test on the innovations, enabling Chernoff-consistent decisions. Empirical results on field data from power systems (BESS and UTK) and synthetic datasets demonstrate that IAE achieves superior or competitive anomaly detection performance (AUROC values up to 0.96) and robust independence properties compared with various baselines such as OCSVM, ANICA, and f-AnoGAN. The approach provides a practical, data-driven pathway to Wiener-style signature extraction for real-time monitoring and anomaly detection in complex, unknown-temporal-structure processes.

Abstract

An innovations sequence of a time series is a sequence of independent and identically distributed random variables with which the original time series has a causal representation. The innovation at a time is statistically independent of the history of the time series. As such, it represents the new information contained at present but not in the past. Because of its simple probability structure, an innovations sequence is the most efficient signature of the original. Unlike the principle or independent component analysis representations, an innovations sequence preserves not only the complete statistical properties but also the temporal order of the original time series. An long-standing open problem is to find a computationally tractable way to extract an innovations sequence of non-Gaussian processes. This paper presents a deep learning approach, referred to as Innovations Autoencoder (IAE), that extracts innovations sequences using a causal convolutional neural network. An application of IAE to the one-class anomalous sequence detection problem with unknown anomaly and anomaly-free models is also presented.

Paper Structure

This paper contains 29 sections, 1 theorem, 12 equations, 7 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Summary of results
  3. Notations
  4. Background and Related Work
  5. Innovations representation and estimation
  6. One-class anomalous sequence detection
  7. Innovations Autoencoder
  8. Parameterization and Dimensionality of Innovations Autoencoder
  9. Training of IAE: Dimensionality and the Training Objective
  10. Convergence Analysis
  11. Anomalous Sequence Detection via Innovations Autoencoder
  12. A Nonparametric Anomaly Model
  13. Anomaly detection via IAE and Uniformity Test
  14. Performance Evaluation
  15. Training and testing datasets
  16. ...and 14 more sections

Key Result

Theorem 1

Let ${\cal A}_{m}^{(N)} = (G_{\theta_m^{\tiny (N)}},H_{\theta_m^{\tiny (N)}})$ be the $m$-dimensional autoencoder optimally trained with training sample dimension $N$ according to (eq:IGAN). Under (A1-A4), ${\cal A}_{m}^{(N)}$ converges (in finite block size $N$) to ${\cal A}$.

Figures (7)

  • Figure 1: Structure of IAE (left) and a causal CNN implementation (right).
  • Figure 2: IAE Uniformity Test. Top: Implementation schematic and test statistics under ${\cal H}_0$. Bottom left: Histogram of $(x_t)$ and $(\nu_t)$ under ${\cal H}_0$ and ${\cal H}_1$. Bottom right: an example of coincidence statistics $(T_i)$ with 30 quantization levels and 15 samples.
  • Figure 3: Battery Energy System at EPFL. See SossanFabrizioNamor_2016 for detailed description.
  • Figure 4: Detection performance for the BESS dataset. Left: ROC curves. (AUROC: IAE:0.8354, ANICA:0.5027 OCSVM:0.4903 F-AnoGAN:0.4993) Right: Anomaly and anomaly-free traces.
  • Figure 5: Detection performance for the UTK dataset. Left: ROC curves. (AUROC: IAE:0.9135, ANICA:0.5967, OCSMV:0.2978, F-AnoGAN:0.4393) Right: Anomaly and anomaly-free traces.
  • ...and 2 more figures

Theorems & Definitions (2)

  • Definition 1: Finite training-block convergence
  • Theorem 1