Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maximum Mean Discrepancy on Exponential Windows for Online Change Detection

Florian Kalinke, Marco Heyden, Georg Gntuni, Edouard Fouché, Klemens Böhm

TL;DR

A new change detection algorithm, called Maximum Mean Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows is proposed and it is proved that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and it outperforms the state of the art on benchmark data streams.

Abstract

Detecting changes is of fundamental importance when analyzing data streams and has many applications, e.g., in predictive maintenance, fraud detection, or medicine. A principled approach to detect changes is to compare the distributions of observations within the stream to each other via hypothesis testing. Maximum mean discrepancy (MMD), a (semi-)metric on the space of probability distributions, provides powerful non-parametric two-sample tests on kernel-enriched domains. In particular, MMD is able to detect any disparity between distributions under mild conditions. However, classical MMD estimators suffer from a quadratic runtime complexity, which renders their direct use for change detection in data streams impractical. In this article, we propose a new change detection algorithm, called Maximum Mean Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows. We prove that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and show empirically that it outperforms the state of the art on benchmark data streams.

Maximum Mean Discrepancy on Exponential Windows for Online Change Detection

TL;DR

A new change detection algorithm, called Maximum Mean Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows is proposed and it is proved that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and it outperforms the state of the art on benchmark data streams.

Abstract

Detecting changes is of fundamental importance when analyzing data streams and has many applications, e.g., in predictive maintenance, fraud detection, or medicine. A principled approach to detect changes is to compare the distributions of observations within the stream to each other via hypothesis testing. Maximum mean discrepancy (MMD), a (semi-)metric on the space of probability distributions, provides powerful non-parametric two-sample tests on kernel-enriched domains. In particular, MMD is able to detect any disparity between distributions under mild conditions. However, classical MMD estimators suffer from a quadratic runtime complexity, which renders their direct use for change detection in data streams impractical. In this article, we propose a new change detection algorithm, called Maximum Mean Discrepancy on Exponential Windows (MMDEW), that combines the benefits of MMD with an efficient computation based on exponential windows. We prove that MMDEW enjoys polylogarithmic runtime and logarithmic memory complexity and show empirically that it outperforms the state of the art on benchmark data streams.
Paper Structure (26 sections, 7 theorems, 27 equations, 9 figures, 3 tables)

This paper contains 26 sections, 7 theorems, 27 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Definitions and background
  4. Our proposed algorithm
  5. Threshold for the hypothesis test
  6. Proposed data structure
  7. Properties
  8. Insertion of observations
  9. MMD computation and change detection
  10. MMDEW Algorithm
  11. Experiments
  12. Synthetic data
  13. ARL and MTD.
  14. Runtime.
  15. Real-world classification data
  16. ...and 11 more sections

Key Result

Proposition 1

Let $\mathbb{P}, \mathbb{Q} \in \mathcal{M}_1^+(\mathcal{X})$, $\hat{\mathbb{P}}_m = \{x_1,\ldots,x_m\}\stackrel{\text{i.i.d.}}{\sim} \mathbb{P}$, $\hat{\mathbb{Q}}_n = \{y_1,\ldots,y_n\}\stackrel{\text{i.i.d.}}{\sim} \mathbb{Q}$. Assume that $0\leq k(x,y) \leq K$ for all $x,y \in \mathcal{X}$ and $

Figures (9)

  • Figure 1: Schematic representation of Example \ref{['ex:example-1']}. For a given step, the proposed scheme stores the windows in bold face.
  • Figure 1: Proposed MMDEW change detection algorithm.
  • Figure 2: Set up of data structure with subsampling upon inserting $x_1,\dots,x_6$. MMDEW stores the windows in bold face at the end of the merge operations. Observations $x_2$ and $x_3$ are not stored explicitly due to the sampling applied. $x_4$ is split into two lines for readability. See Example \ref{['example:subsampling']} for a detailed discussion.
  • Figure 3: Average run length (ARL) and expected detection delay / mean time to detection (EDD / MTD) of MMDEW on synthetically generated data.
  • Figure 4: Comparison of runtimes per insert operation (l.h.s.) and least squares fit validating the theoretical runtime complexity of MMDEW w.r.t. the runtime observed in practice (r.h.s.).
  • ...and 4 more figures

Theorems & Definitions (14)

  • Proposition 1
  • Remark 1
  • Lemma 1
  • Lemma 2
  • proof
  • Example 1
  • Proposition 2
  • proof
  • Proposition 3
  • Remark 2
  • ...and 4 more