Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Normalizing self-supervised learning for provably reliable Change Point Detection

Alexandra Bazarova, Evgenia Romanenkova, Alexey Zaytsev

TL;DR

This work tackles unsupervised change point detection by unifying self-supervised time-series representation learning with spectral normalization to guarantee reliability. The authors prove that spectral normalization preserves test power for both kernel-based and likelihood-ratio CPD tests, leveraging bi-Lipschitz and kernel-distance preservation properties. They instantiate the framework with two SSL backbones, TS2Vec and TS-BYOL, and validate improvements on Yahoo! A4 Benchmark, USC-HAD, and HASC datasets, achieving competitive or leading F1 scores. The approach offers a theoretically grounded and practically effective path to robust CPD in high-dimensional time series, with notable gains in detection robustness and sensitivity. Overall, this work bridges representation learning and classical CPD theory to deliver more reliable, scalable CPD in real-world streams.

Abstract

Change point detection (CPD) methods aim to identify abrupt shifts in the distribution of input data streams. Accurate estimators for this task are crucial across various real-world scenarios. Yet, traditional unsupervised CPD techniques face significant limitations, often relying on strong assumptions or suffering from low expressive power due to inherent model simplicity. In contrast, representation learning methods overcome these drawbacks by offering flexibility and the ability to capture the full complexity of the data without imposing restrictive assumptions. However, these approaches are still emerging in the CPD field and lack robust theoretical foundations to ensure their reliability. Our work addresses this gap by integrating the expressive power of representation learning with the groundedness of traditional CPD techniques. We adopt spectral normalization (SN) for deep representation learning in CPD tasks and prove that the embeddings after SN are highly informative for CPD. Our method significantly outperforms current state-of-the-art methods during the comprehensive evaluation via three standard CPD datasets.

Normalizing self-supervised learning for provably reliable Change Point Detection

TL;DR

This work tackles unsupervised change point detection by unifying self-supervised time-series representation learning with spectral normalization to guarantee reliability. The authors prove that spectral normalization preserves test power for both kernel-based and likelihood-ratio CPD tests, leveraging bi-Lipschitz and kernel-distance preservation properties. They instantiate the framework with two SSL backbones, TS2Vec and TS-BYOL, and validate improvements on Yahoo! A4 Benchmark, USC-HAD, and HASC datasets, achieving competitive or leading F1 scores. The approach offers a theoretically grounded and practically effective path to robust CPD in high-dimensional time series, with notable gains in detection robustness and sensitivity. Overall, this work bridges representation learning and classical CPD theory to deliver more reliable, scalable CPD in real-world streams.

Abstract

Change point detection (CPD) methods aim to identify abrupt shifts in the distribution of input data streams. Accurate estimators for this task are crucial across various real-world scenarios. Yet, traditional unsupervised CPD techniques face significant limitations, often relying on strong assumptions or suffering from low expressive power due to inherent model simplicity. In contrast, representation learning methods overcome these drawbacks by offering flexibility and the ability to capture the full complexity of the data without imposing restrictive assumptions. However, these approaches are still emerging in the CPD field and lack robust theoretical foundations to ensure their reliability. Our work addresses this gap by integrating the expressive power of representation learning with the groundedness of traditional CPD techniques. We adopt spectral normalization (SN) for deep representation learning in CPD tasks and prove that the embeddings after SN are highly informative for CPD. Our method significantly outperforms current state-of-the-art methods during the comprehensive evaluation via three standard CPD datasets.

Paper Structure

This paper contains 34 sections, 5 theorems, 27 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Methodology
  4. Spectral Normalization preserves the CPD quality
  5. Bi-Lipschitz neural networks
  6. Preservation of kernel distances
  7. Kernel-based test
  8. Likelihood ratio tests
  9. General pipeline
  10. Models
  11. TS2Vec
  12. TS-BYOL
  13. Results
  14. Compared methods
  15. Datasets
  16. ...and 19 more sections

Key Result

Lemma 3.1

sngp Consider a hidden mapping $h: \mathcal{X} \rightarrow \mathcal{H}$ of a form resblock. If for $0 < \alpha \leq 1$ all $g_l$'s are $\alpha$-Lipschitz, i.e., $\lVert g_l(\mathbf{X}) - g_l(\mathbf{X}') \rVert_{\mathcal{H}} \leq \alpha \lVert \mathbf{X} - \mathbf{X}' \rVert_{\mathcal{X}} \; \forall where $L_1 = (1 - \alpha)^L, \, L_2 = (1 + \alpha)^L$.

Figures (8)

  • Figure 1: The structure of the proposed change point detection procedure based on a model obtained via self-supervised learning with spectral normalization.
  • Figure 2: Cosine distances between the subsequent subintervals of observations. The value above a predefined threshold indicates the presence of a changepoint. The color filled area denotes that the corresponding intervals contain a change point.
  • Figure 3: TS2Vec architecture. The upper part of the figure is the hierarchical contrasting procedure; the lower part is the selection of positive and negative pairs.
  • Figure 4: BYOL's architecture. The upper part is the training Online network, and the lower part is the Target network with disabled backpropagation.
  • Figure 5: Comparison of TS2Vec performance with and without spectral normalization on all datasets. The cosine distance was used as the test statistic. The upper row --- Yahoo!A4Benchmark, middle --- HASC, lower --- USC-HAD.
  • ...and 3 more figures

Theorems & Definitions (5)

  • Lemma 3.1
  • Lemma 3.2
  • Theorem 1
  • Proposition 3.1
  • Theorem 2