C-SHAP for time series: An approach to high-level temporal explanations
Annemarie Jutte, Faizan Ahmed, Jeroen Linssen, Maurice van Keulen
TL;DR
This paper introduces C-SHAP, a concept-based extension of SHAP for time series, enabling explanations in terms of high-level patterns rather than point-level attributions. It defines concepts as higher-level features $C$ and derives Shapley-based contributions $\phi_i$ for each concept, providing a formal framework that generalizes to univariate and multivariate time series. The implementation relies on time series decomposition (Prophet) to construct concepts such as Growth, Daily, Weekly, and Other, and uses a masking procedure to estimate each concept's impact on model outputs. A proof-of-concept demonstration on hourly energy consumption forecasting shows that Growth dominates global explanations, with local explanations clarifying how specific patterns drive predictions, while identifying limitations and future extensions for richer concept sets and non-stationary concept behavior.
Abstract
Time series are ubiquitous in domains such as energy forecasting, healthcare, and industry. Using AI systems, some tasks within these domains can be efficiently handled. Explainable AI (XAI) aims to increase the reliability of AI solutions by explaining model reasoning. For time series, many XAI methods provide point- or sequence-based attribution maps. These methods explain model reasoning in terms of low-level patterns. However, they do not capture high-level patterns that may also influence model reasoning. We propose a concept-based method to provide explanations in terms of these high-level patterns. In this paper, we present C-SHAP for time series, an approach which determines the contribution of concepts to a model outcome. We provide a general definition of C-SHAP and present an example implementation using time series decomposition. Additionally, we demonstrate the effectiveness of the methodology through a use case from the energy domain.