Delta-XAI: A Unified Framework for Explaining Prediction Changes in Online Time Series Monitoring

TL;DR

<3-5 sentence high-level summary> Delta-XAI introduces a unified framework to explain prediction changes in online time series by reusing existing single-time XAI methods through a prediction-difference wrapper and by proposing SWING, an IG-based method that accounts for temporal dynamics via retrospective baselines, dual-path integration, and piecewise-linear histories. The authors provide a comprehensive evaluation suite tailored to online settings and demonstrate that classic gradient-based explanations can outperform newer methods when adapted to temporal data. Across diverse real-world and synthetic benchmarks, SWING achieves superior fidelity, sufficiency, and coherence while maintaining reasonable efficiency. This work thus shifts the emphasis from static explanations to dynamic, time-sensitive attributions, enhancing trustworthiness in high-stakes domains such as healthcare and finance.

Abstract

Explaining online time series monitoring models is crucial across sensitive domains such as healthcare and finance, where temporal and contextual prediction dynamics underpin critical decisions. While recent XAI methods have improved the explainability of time series models, they mostly analyze each time step independently, overlooking temporal dependencies. This results in further challenges: explaining prediction changes is non-trivial, methods fail to leverage online dynamics, and evaluation remains difficult. To address these challenges, we propose Delta-XAI, which adapts 14 existing XAI methods through a wrapper function and introduces a principled evaluation suite for the online setting, assessing diverse aspects, such as faithfulness, sufficiency, and coherence. Experiments reveal that classical gradient-based methods, such as Integrated Gradients (IG), can outperform recent approaches when adapted for temporal analysis. Building on this, we propose Shifted Window Integrated Gradients (SWING), which incorporates past observations in the integration path to systematically capture temporal dependencies and mitigate out-of-distribution effects. Extensive experiments consistently demonstrate the effectiveness of SWING across diverse settings with respect to diverse metrics. Our code is publicly available at https://anonymous.4open.science/r/Delta-XAI.

Figures (10)

• Figure 1: Motivation for explaining prediction changes through illustrative scenarios that are not generated by actual XAI outputs. (top) Vital signs across $T_1, T_2, T_3$: risk rises from 10% to 90% then partially recovers (70%). Conventional attribution at $T_3$ misleads, while our method highlights features driving recovery. (bottom) Risk evolves from 10% at $T_1$ to 80% at $T_2$ and slightly increases (85%) at $T_3$ due to delayed effects. Our method attributes prediction changes between $T_1 \rightarrow T_2$ and $T_2 \rightarrow T_3$, resolving these issues.
• Figure 2: Overview of the proposed SWING framework for explaining prediction changes in online patient monitoring. SWING extends conventional Integrated Gradients (IG) by replacing zero-baseline straight paths with line integrals over shifted historical windows and piecewise-linear paths, capturing temporal dynamics and mitigating out-of-distribution effects.
• Figure 3: Computational efficiency analysis comparing SWING with baselines on the MIMIC-III benchmark. (a) Elapsed real time per sample (sec/sample, log-scale) versus AUPD ($K = 50$). (b) GPU peak memory consumption per sample (MB/sample) versus AUPD ($K = 50$).
• Figure 4: Hyperparameter sensitivity of SWING with respect to $n_{\text{samples}}$, compared to IG (dotted gray line).
• Figure 5: Qualitative case study showing attributions extracted with XAI methods on MIMIC-III mimic3 using LSTM lstm backbone with $T_1 = 47$ and $T_2 = 48$, i.e., $T_2 - T_1 = 1$. The uppermost-left heatmap displays the normalized input features, while the remaining fifteen panels illustrate the attribution heatmaps generated by each XAI method under the Delta-XAI framework, reflecting their respective explanations of the score changes.

Theorems & Definitions (6)

• Theorem 1: Attribution Decomposition Theorem for Online Completeness
• Theorem 2: Online Completeness
• Theorem 3: Implementation Invariance
• Theorem 4: Skew-Symmetry
• Lemma 5: Completeness of Integrated Gradients along General Paths
• proof