Mechanistic Interpretability for Transformer-based Time Series Classification

TL;DR

Transformer-based time-series classifiers achieve state-of-the-art accuracy but remain opaque to internal decision processes. The paper advances Mechanistic Interpretability for Time Series Transformers by adapting Activation Patching, Attention Saliency, and Sparse Autoencoders to reveal causal information flow from input timesteps through attention heads to class logits and by constructing causal graphs. Key findings show that early Transformer layers carry the strongest causal signals, a few heads dominate the internal computation, and sparse latent motifs align with class-specific temporal patterns, offering tangible interpretability gains. These insights support more trustworthy deployment of time-series models in safety-critical domains and guide future development of MI tools for sequential architectures.

Abstract

Transformer-based models have become state-of-the-art tools in various machine learning tasks, including time series classification, yet their complexity makes understanding their internal decision-making challenging. Existing explainability methods often focus on input-output attributions, leaving the internal mechanisms largely opaque. This paper addresses this gap by adapting various Mechanistic Interpretability techniques; activation patching, attention saliency, and sparse autoencoders, from NLP to transformer architectures designed explicitly for time series classification. We systematically probe the internal causal roles of individual attention heads and timesteps, revealing causal structures within these models. Through experimentation on a benchmark time series dataset, we construct causal graphs illustrating how information propagates internally, highlighting key attention heads and temporal positions driving correct classifications. Additionally, we demonstrate the potential of sparse autoencoders for uncovering interpretable latent features. Our findings provide both methodological contributions to transformer interpretability and novel insights into the functional mechanics underlying transformer performance in time series classification tasks.

Figures (18)

• Figure 1: Confusion matrix of the trained TST on the JapaneseVowels test set. Rows are true classes; columns are predicted classes.
• Figure 2: Change in true‐class probability ($\Delta P$) when patching all heads of each encoder layer.
• Figure 3: Head‐level activation patching: true‐class probability change $\Delta P$ after patching each individual attention head.
• Figure 4: True‐class probability $\Delta P_t$ after patching each timestep of Layer 0 Head 6.
• Figure 5: Attention saliency scores of Layer 0 Head 6 overlaid on the raw input series for the clean (top) and corrupt (bottom) instances.