Concept Extraction for Time Series with ECLAD-ts

TL;DR

The paper tackles the challenge of explaining CNN-based time series classifiers by introducing ECLAD-ts, a post-hoc global Concept Extraction method tailored to multivariate time series. It builds on the ECLAD framework by encoding latent space with Local Aggregated Descriptors (LADs), clustering these representations to form concepts, and computing a channel-aware relevance score based on model gradients, enabling both temporal and channel-wise localization. Through synthetic datasets with predefined primitives and real-world datasets like GunPoint and P2S, the authors demonstrate that ECLAD-ts yields higher representation and importance correctness than baselines (ECLAD, ConceptShap, MultiVISION), and effectively reveals shortcut learning and the features driving predictions. The approach offers practical value for model transparency, enabling practitioners to inspect which patterns and channels most influence decisions, thereby improving trust, debugging, and deployment in safety-critical domains. It also highlights limitations and future directions, such as visualization challenges and hyperparameter sensitivity, guiding further refinement of time-series explainability methods.

Abstract

Convolutional neural networks (CNNs) for time series classification (TSC) are being increasingly used in applications ranging from quality prediction to medical diagnosis. The black box nature of these models makes understanding their prediction process difficult. This issue is crucial because CNNs are prone to learning shortcuts and biases, compromising their robustness and alignment with human expectations. To assess whether such mechanisms are being used and the associated risk, it is essential to provide model explanations that reflect the inner workings of the model. Concept Extraction (CE) methods offer such explanations, but have mostly been developed for the image domain so far, leaving a gap in the time series domain. In this work, we present a CE and localization method tailored to the time series domain, based on the ideas of CE methods for images. We propose the novel method ECLAD-ts, which provides post-hoc global explanations based on how the models encode subsets of the input at different levels of abstraction. For this, concepts are produced by clustering timestep-wise aggregations of CNN activation maps, and their importance is computed based on their impact on the prediction process. We evaluate our method on synthetic and natural datasets. Furthermore, we assess the advantages and limitations of CE in time series through empirical results. Our results show that ECLAD-ts effectively explains models by leveraging their internal representations, providing useful insights about their prediction process.

Figures (11)

• Figure 1: Validation Pipeline for Concept Extraction. (1) Primitives definition. (2) Synthetic dataset generation. (3) Models are trained on the synthetic dataset and concept extraction is performed. (4) Extracted concepts are localized and compared with the ground truth primitives to establish associations. (5) Evaluation metrics for representation and importance correctness are computed.
• Figure 2: Left: Examples of class 0 and 1 of syntheticL2 dataset. Right: Examples of class 0 and 1 of syntheticL4 dataset. The red boxes signalize the primitives used for generating the samples of each class.
• Figure 3: Examples of classes 0, 1, and 2 of syntheticLm dataset. The red boxes signalize the primitives used for generating the samples of each class.
• Figure 4: Concept extraction from ResNet18 on the syntheticL2 dataset is illustrated for four methods. The dataset consists of one channel with a primary primitive $p_0$ and an uninterrupted background $p_1$, both serving as discriminative cues. The model (seed=1) achieved a validation accuracy of $100\%$. In each panel, rows denote individual concepts and columns represent instances (with headers showing actual/predicted labels: the first two for class 0 and the latter two for class 1), highlighted regions indicate where concepts appear, and left labels report their importance scores. Notably, ECLAD-ts and ECLAD extract $p_0$-related concepts in $c_1$ and $c_2$, respectively, while ConceptShap and MultiVISION capture background cues.
• Figure 5: Boxplots of (left) representation correctness and (right) importance correctness for CE methods on the syntheticL2 dataset. The box plots aggregate the data across all random seeds and the concept numbers chosen as a hyperparameter. All models achieved a validation accuracy of $100\%$. In both plots higher is better, as it means that the extracted concepts are more aligned with the ideal results of a concept extraction method. The plots that are collapsed at $-0.200$ are methods that exclusively obtained the maximum penalty by failing to identify alignment. ECLAD-based methods consistently achieve higher representation correctness, while ECLAD-ts attains the best importance correctness overall, highlighting its ability to extract concepts which are aligned with the underlying primitives.