Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretraining

Qisen Cheng, Jinming Xing, Chang Xue, Xiaoran Yang

TL;DR

ShapTST tackles the computational burden of generating Shapley-value explanations for time-series transformers by integrating explanation generation into a single forward pass through a Shapley-based pre-training framework. It augments a time-series Transformer with an explainer head and uses multi-level masking, InfoNCE-based contrastive pre-training, and Shapley-based regularization to learn robust representations that support explanations and predictions simultaneously. The approach achieves competitive predictive performance, faithful explanations comparable to TimeSHAP, and a 4–5× speedup in explanation generation, enhancing practical deployment in safety-critical settings. Overall, ShapTST provides an efficient, explainable, and robust framework for time-series analysis that combines prediction and interpretation in a unified model.

Abstract

In this paper, we propose ShapTST, a framework that enables time-series transformers to efficiently generate Shapley-value-based explanations alongside predictions in a single forward pass. Shapley values are widely used to evaluate the contribution of different time-steps and features in a test sample, and are commonly generated through repeatedly inferring on each sample with different parts of information removed. Therefore, it requires expensive inference-time computations that occur at every request for model explanations. In contrast, our framework unifies the explanation and prediction in training through a novel Shapley-based pre-training design, which eliminates the undesirable test-time computation and replaces it with a single-time pre-training. Moreover, this specialized pre-training benefits the prediction performance by making the transformer model more effectively weigh different features and time-steps in the time-series, particularly improving the robustness against data noise that is common to raw time-series data. We experimentally validated our approach on eight public datasets, where our time-series model achieved competitive results in both classification and regression tasks, while providing Shapley-based explanations similar to those obtained with post-hoc computation. Our work offers an efficient and explainable solution for time-series analysis tasks in the safety-critical applications.

Unifying Prediction and Explanation in Time-Series Transformers via Shapley-based Pretraining

TL;DR

ShapTST tackles the computational burden of generating Shapley-value explanations for time-series transformers by integrating explanation generation into a single forward pass through a Shapley-based pre-training framework. It augments a time-series Transformer with an explainer head and uses multi-level masking, InfoNCE-based contrastive pre-training, and Shapley-based regularization to learn robust representations that support explanations and predictions simultaneously. The approach achieves competitive predictive performance, faithful explanations comparable to TimeSHAP, and a 4–5× speedup in explanation generation, enhancing practical deployment in safety-critical settings. Overall, ShapTST provides an efficient, explainable, and robust framework for time-series analysis that combines prediction and interpretation in a unified model.

Abstract

In this paper, we propose ShapTST, a framework that enables time-series transformers to efficiently generate Shapley-value-based explanations alongside predictions in a single forward pass. Shapley values are widely used to evaluate the contribution of different time-steps and features in a test sample, and are commonly generated through repeatedly inferring on each sample with different parts of information removed. Therefore, it requires expensive inference-time computations that occur at every request for model explanations. In contrast, our framework unifies the explanation and prediction in training through a novel Shapley-based pre-training design, which eliminates the undesirable test-time computation and replaces it with a single-time pre-training. Moreover, this specialized pre-training benefits the prediction performance by making the transformer model more effectively weigh different features and time-steps in the time-series, particularly improving the robustness against data noise that is common to raw time-series data. We experimentally validated our approach on eight public datasets, where our time-series model achieved competitive results in both classification and regression tasks, while providing Shapley-based explanations similar to those obtained with post-hoc computation. Our work offers an efficient and explainable solution for time-series analysis tasks in the safety-critical applications.
Paper Structure (18 sections, 7 equations, 5 figures, 3 tables)

This paper contains 18 sections, 7 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. ShapTST
  4. Mask Design
  5. Time level masking
  6. Feature level masking
  7. Cell level masking
  8. Shapley-based pre-training
  9. Shapley value-based regularization
  10. Experiment
  11. Model prediction performance
  12. Model prediction robustness
  13. Model explanations
  14. Time level explanation
  15. Feature level explanation
  16. ...and 3 more sections

Figures (5)

  • Figure 1: The ShapTST framework. It involves 1) a Shaley-based pre-training stage, where TST is trained on both original and perturbed data to learn robust representations and capability to predict with partial information, and 2) a fine-tuning stage that enables TST to generate predictions alongside explanations.
  • Figure 2: Maskings for the training of ShapTST
  • Figure 3: Performance on AReM dataset with added noise. The blue line is trained without Shapley-based regularization while the red line is trained with Shapley regularization
  • Figure 4: Time level explanation example of AReM Dataset. Adjacent 4 time points are aggregated into one time step.
  • Figure 5: Time and feature level explanation examples (AReM). Adjacent 4 time points are aggregated into one time step.