TimeSAE: Sparse Decoding for Faithful Explanations of Black-Box Time Series Models

TL;DR

TimeSAE tackles the challenge of explaining black-box time-series models by learning an end-to-end sparse autoencoder that decomposes inputs into interpretable concepts. It integrates causal reasoning through approximated counterfactuals and the Causal Concept Effect to ensure faithfulness and robustness to distribution shifts, while introducing compositionality via a decompositional decoder and concept masks. The approach combines JumpReLU-based sparsity, temporal convolutional backbones, and CAR-aligned concept discovery to produce faithful, localized explanations that generalize beyond training distribution. Empirical results on eight datasets—including a new EliteLJ sports dataset—show TimeSAE outperforms baselines in faithfulness and distributional alignment, with favorable inference speed and scalable hyperparameters. The work advances practical explainability for time-series models and opens avenues for white-box concept-based modeling and deeper interpretability analyses.

Abstract

As black box models and pretrained models gain traction in time series applications, understanding and explaining their predictions becomes increasingly vital, especially in high-stakes domains where interpretability and trust are essential. However, most of the existing methods involve only in-distribution explanation, and do not generalize outside the training support, which requires the learning capability of generalization. In this work, we aim to provide a framework to explain black-box models for time series data through the dual lenses of Sparse Autoencoders (SAEs) and causality. We show that many current explanation methods are sensitive to distributional shifts, limiting their effectiveness in real-world scenarios. Building on the concept of Sparse Autoencoder, we introduce TimeSAE, a framework for black-box model explanation. We conduct extensive evaluations of TimeSAE on both synthetic and real-world time series datasets, comparing it to leading baselines. The results, supported by both quantitative metrics and qualitative insights, show that TimeSAE provides more faithful and robust explanations. Our code is available in an easy-to-use library TimeSAE-Lib: https://anonymous.4open.science/w/TimeSAE-571D/.

Figures (14)

• Figure 1: Overview of Time Series Sparse Autoencoder ( TimeSAE ):(A) The framework assumes access to a black-box model $f$ and aims to explain its predictions on data ${\mathbf{x}} \in {\mathcal{X}}$ by learning an explainer ${\mathcal{E}}$ and a decoder $\boldsymbol{g}$ that decompose the time series into interpretable components. (B) For faithfulness, the sparse autoencoder $({\mathcal{E}}, \boldsymbol{g})$ incorporates properties to leverage counterfactual explanations. A contrastive learning loss incorporates a set of counterfactuals $\tilde{{\mathbf{x}}}^{cf}$, obtained by intervening on concepts and which produce, via $f$, a contradictory label $\tilde{{\mathbf{y}}}^{cf}$ relative to the original $\tilde{{\mathbf{y}}}$. (C) To ensure compositional explanations, the method enforces the explainer ${\mathcal{E}}$ to generate consistent explanations by combining intermediate findings.
• Figure 2: Explanation performance on all datasets and metrics (AUPRC, AUP, AUR). Higher is better. The rightmost panel shows average scores. Methods are ranked left to right from worst to best.
• Figure 3: AUPRC explanation performance (higher is better) across methods for each dataset.
• Figure 4: Spearman correlation ($\rho \approx -0.981$) between the Faithfulness metric (${\mathcal{F}}_{\mathbf{x}}$) and the Counterfactual Approximation Error ($\epsilon_{cf}$).
• Figure 5: (a) Examples of explanation compared to the ground truth on the FreqShapes dataset. (b) Effects of excluding the Concepts Consistency and Counterfactual term on the Faithfulness metric ${\mathcal{F}}_{{\mathbf{x}}}$, with standard deviations shown over 10 runs. Using counterfactuals produces more faithful explanations. (c) Effect of sparsity on reconstruction fidelity. Increasing sparsity generally improves interpretability; however, excessive sparsity can compromise fidelity.

Theorems & Definitions (8)

• Definition 1: Faithful Explanations.
• Definition 2: CaCE CausalConceptEffect2019
• Definition 3: Approximated Counterfactual Explanation gat2023faithful
• Theorem 1: Faithfulness in Sparse Autoencoder-Based Approximate Counterfactuals
• Definition 3: CaCE CausalConceptEffect2019
• Definition 3: Approximated Counterfactual Explanation gat2023faithful
• Theorem 1: Faithfulness in Sparse Autoencoder-Based Approximate Counterfactuals
• proof