Finding the DeepDream for Time Series: Activation Maximization for Univariate Time Series

TL;DR

The paper tackles interpretability for univariate time-series models by adapting Activation Maximization into Sequence Dreaming, a framework that visualizes temporal patterns learned by neural networks. By incorporating a spectrum of regularization strategies—including loss-based terms, smoothing, and Gaussian filtering—the method generates plausible time-series stimuli that maximize target neuron activations. Experiments on the FordA predictive maintenance dataset demonstrate the ability to produce both centered and border activations, offering insights into the temporal features driving model decisions. The work advances model transparency in critical domains and provides a flexible toolkit for exploring temporal representations learned by deep networks.

Abstract

Understanding how models process and interpret time series data remains a significant challenge in deep learning to enable applicability in safety-critical areas such as healthcare. In this paper, we introduce Sequence Dreaming, a technique that adapts Activation Maximization to analyze sequential information, aiming to enhance the interpretability of neural networks operating on univariate time series. By leveraging this method, we visualize the temporal dynamics and patterns most influential in model decision-making processes. To counteract the generation of unrealistic or excessively noisy sequences, we enhance Sequence Dreaming with a range of regularization techniques, including exponential smoothing. This approach ensures the production of sequences that more accurately reflect the critical features identified by the neural network. Our approach is tested on a time series classification dataset encompassing applications in predictive maintenance. The results show that our proposed Sequence Dreaming approach demonstrates targeted activation maximization for different use cases so that either centered class or border activation maximization can be generated. The results underscore the versatility of Sequence Dreaming in uncovering salient temporal features learned by neural networks, thereby advancing model transparency and trustworthiness in decision-critical domains.

Figures (1)

• Figure 1: General approach split up into two pipelines. First, a projection of the activations of the data into 2D to find interesting regions to focus on. Second, the asteroid12Sequence Dreaming approach generates time series in selected regions of interest to find salient features of the model.