Entropic Analysis of Time Series through Kernel Density Estimation

TL;DR

This paper introduces a KDE-based entropic framework for time series analysis built on Takens' embeddings, enabling multiscale entropy assessment and KL-based change detection. By transforming sequences into 2D point clouds and estimating PDFs with Gaussian KDE, it defines Kernel Density Estimate Entropy (KDEE) and a multiscale ΔKE metric to quantify complexity, alongside a Sliding Baseline Change Detection protocol using a symmetrized KL divergence. The approach is validated across RF signal injection detection, ECG ventricular fibrillation detection, and chaotic-dynamics state detection, demonstrating competitive performance with significantly lower training requirements and broad applicability. The results highlight the method's potential for robust, interpretable change detection in diverse scientific domains.

Abstract

This work presents a novel framework for time series analysis using entropic measures based on the kernel density estimate (KDE) of the time series' Takens' embeddings. Using this framework we introduce two distinct analytical tools: (1) a multi-scale KDE entropy metric, denoted as $Δ\text{KE}$, which quantifies the evolution of time series complexity across different scales by measuring certain entropy changes, and (2) a sliding baseline method that employs the Kullback-Leibler (KL) divergence to detect changes in time series dynamics through changes in KDEs. The $Δ{\rm KE}$ metric offers insights into the information content and ``unfolding'' properties of the time series' embedding related to dynamical systems, while the KL divergence-based approach provides a noise and outlier robust approach for identifying time series change points (injections in RF signals, e.g.). We demonstrate the versatility and effectiveness of these tools through a set of experiments encompassing diverse domains. In the space of radio frequency (RF) signal processing, we achieve accurate detection of signal injections under varying noise and interference conditions. Furthermore, we apply our methodology to electrocardiography (ECG) data, successfully identifying instances of ventricular fibrillation with high accuracy. Finally, we demonstrate the potential of our tools for dynamic state detection by accurately identifying chaotic regimes within an intermittent signal. These results show the broad applicability of our framework for extracting meaningful insights from complex time series data across various scientific disciplines.

Figures (8)

• Figure 1: Sinusoidal time series with additive Gaussian noise (left), two-dimensional Takens' embedding of the time series (center), and resulting kernel density estimation of the Takens' embedding (right).
• Figure 2: KDEE evolution across time scales $\tau$ for structured signal (left) versus noise (right), showing characteristic unfolding behavior.
• Figure 3: Change detection using KL divergence between median baseline (red windows) and analysis window (blue). Bottom panel shows detection statistic with threshold.
• Figure 4: Correlation between SNR and $\Delta\text{KE}$ across modulation types. For each SNR, 10 signals of each of 14 modulation types were generated. Results show means with one standard deviation on a log scale, demonstrating consistent correlation independent of modulation scheme.
• Figure 5: RF signal injection detection example showing background interference, noise, sliding baseline (red window, 10 KDEs), and analysis window (blue window). Both KL divergence and $\Delta\text{KE}$ methods show clear response during injection period, with detected region (red) based on modified z-score threshold.