WinTSR: A Windowed Temporal Saliency Rescaling Method for Interpreting Time Series Deep Learning Models

TL;DR

WinTSR addresses explainability in time series forecasting by introducing a local, post-hoc, model-agnostic interpretation method that explicitly models temporal dependencies over a look-back window $L$. It computes a time-relevance score and a feature-relevance score to form an importance matrix via $\boldsymbol{\phi_t}$ with $\phi_{j,l,t} = \Delta^{feature}_{j,l,t} \times \Delta^{time}_{l,t}$, enabling fast interpretation without training a surrogate model. A comprehensive benchmark against ten interpretation methods on five state-of-the-art models and three real-world datasets demonstrates WinTSR’s superior performance in comprehensiveness and sufficiency, and an open-source framework is provided for end-to-end interpretation of time-series transformers and foundation models. This work advances explainability in time-series forecasting by delivering a scalable, general-purpose tool that applies to both classification and regression tasks and supports modern architectures, including time-series foundation models like CALF.

Abstract

Interpreting complex time series forecasting models is challenging due to the temporal dependencies between time steps and the dynamic relevance of input features over time. Existing interpretation methods are limited by focusing mostly on classification tasks, evaluating using custom baseline models instead of the latest time series models, using simple synthetic datasets, and requiring training another model. We introduce a novel interpretation method, \textit{Windowed Temporal Saliency Rescaling (WinTSR)} addressing these limitations. WinTSR explicitly captures temporal dependencies among the past time steps and efficiently scales the feature importance with this time importance. We benchmark WinTSR against 10 recent interpretation techniques with 5 state-of-the-art deep-learning models of different architectures, including a time series foundation model. We use 3 real-world datasets for both time-series classification and regression. Our comprehensive analysis shows that WinTSR significantly outperforms other local interpretation methods in overall performance. Finally, we provide a novel, open-source framework to interpret the latest time series transformers and foundation models.

Figures (3)

• Figure 1: An overview of our framework with the proposed Windowed Temporal Saliency Rescaling (WinTSR) method.
• Figure 2: An illustration of (a) our proposed WinTSR method and (b) evaluating interpretation without ground truth. The above example is about calculating comprehensiveness at k=50% with three steps.
• Figure 3: Interpreted relevance scores (when $\tau=1$ or $o=1$) for an arbitrarily chosen example from each dataset, some selected methods, and the iTransformer model. For MIMIC-III the overall most important features of each method are annotated. Both electricity and traffic features show high importance for more recent features and a 24-hour pattern for most methods.