Enhancing Time-Series Anomaly Detection by Integrating Spectral-Residual Bottom-Up Attention with Reservoir Computing

TL;DR

A spectral residual RC (SR-RC) is proposed that integrates the spectral residual (SR) method - a learning-free, bottom-up attention mechanism - with RC and outperformed conventional RC and logistic-regression models based on values extracted by the SR method across benchmark tasks and real-world time-series datasets.

Abstract

Reservoir computing (RC) establishes the basis for the processing of time-series data by exploiting the high-dimensional spatiotemporal response of a recurrent neural network to an input signal. In particular, RC trains only the output layer weights. This simplicity has drawn attention especially in Edge Artificial Intelligence (AI) applications. Edge AI enables time-series anomaly detection in real time, which is important because detection delays can lead to serious incidents. However, achieving adequate anomaly-detection performance with RC alone may require an unacceptably large reservoir on resource-constrained edge devices. Without enlarging the reservoir, attention mechanisms can improve accuracy, although they may require substantial computation and undermine the learning efficiency of RC. In this study, to improve the anomaly detection performance of RC without sacrificing learning efficiency, we propose a spectral residual RC (SR-RC) that integrates the spectral residual (SR) method - a learning-free, bottom-up attention mechanism - with RC. We demonstrated that SR-RC outperformed conventional RC and logistic-regression models based on values extracted by the SR method across benchmark tasks and real-world time-series datasets. Moreover, because the SR method, similarly to RC, is well suited for hardware implementation, SR-RC suggests a practical direction for deploying RC as Edge AI for time-series anomaly detection.

Figures (5)

• Figure 1: Structure of the Spectral Residual (SR) method and the proposed models. The left side of the figure depicts the processing procedure of the SR method. The SR method applies a Fourier transform to the time series data, computes the spectral residual, and generates a saliency map through an inverse Fourier transform (see Methods for details). As illustrated in the upper right of the figure, the reservoir computing (RC) system comprises an input layer, a reservoir layer, and an output layer. The conventional model inputs only the time series data into the RC. In contrast, the lower-right panel presents the proposed models: one that inputs only the saliency map and a multi-input model that uses both the original time series data and saliency map (see Methods for details).
• Figure 2: Baseline time series data and introduced anomalies. The central part of the figure shows the baseline time series data. Anomalies are introduced at randomly selected points or intervals within this central time series. The bottom-right of the figure illustrates global outliers, which are anomalies introduced by inserting spikes at randomly selected points in the time series. The top-right of the figure depicts contextual outliers, where randomly selected points in the time series deviate from their surrounding values. The bottom-left of the figure shows shapelet outliers, which are introduced by selecting random intervals in the time series and replaced by different waveforms. The top-left of the figure illustrates seasonal outliers, which are created by selecting random intervals in the time series and altering the frequency of the underlying wave.
• Figure 3: Simulation results of SR-RC for point-wise outliers. From top to bottom, the figure displays the one-dimensional time-series data with contextual outliers, the corresponding saliency map computed from this data, the dynamics of 10 neurones in the reservoir layer of the SR-RC model, and the anomaly probabilities predicted by both the SR-RC and SR-Logi models. In this experiment, the anomaly occurrence probability ($\delta$) for contextual outliers was set to 0.05, and the SR-RC model comprised 100 neurones. The reservoir layer exhibited transient fluctuations when spikes were input, depending on the magnitude of the spikes in the saliency map. In the bottom panel, asterisks denote cases in which SR-RC predicts a high anomaly probability for subtle spikes in the saliency map, whereas the logistic-regression model does not.
• Figure 4: $\mathbf{a}$ Performance comparison on the benchmark task when the baseline time series is a single sine wave. $\mathbf{b}$ Performance comparison when the baseline is the sum of four sine waves. $\mathbf{c}$ Performance comparison when the baseline is the sum of four sine waves with irrational frequency ratios. For $\mathbf{a}$–$\mathbf{c}$, the x-axis shows the anomaly probability at each time step; the mean F1 score is averaged over 10 runs, and error bars denote the standard error. $\mathbf{d}$ For the four-sine baseline, the mean F1 score obtained with RC as a function of reservoir size. The mean is averaged over 10 runs, and error bars denote the standard error. Horizontal dashed lines (light blue and red, respectively) indicate the mean F1 scores of Multi-SR-RC and SR-RC at a fixed reservoir size of 100. In $\mathbf{a}$–$\mathbf{d}$, from top to bottom, the baseline time series is injected with seasonal, shapelet, contextual, and global outliers.
• Figure 5: Overview of the Yahoo! Webscope S5 dataset used for real-world time-series anomaly-detection tasks. The dataset is divided into four benchmarks (A1–A4). The A1 benchmark contains hourly traffic measurements from a web service; the figure marks manually labeled anomalies with red circles. From the 67 datasets in A1, we selected six time series in which a hold-out split—training $49\%$, validation $21\%$, and test $30\%$—contains at least one anomaly in every subset. Each panel plots training data in orange, validation data in blue, and test data in green. Task numbers match the IDs assigned in the A1 benchmark.