Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Causal Interventional Prediction System for Robust and Explainable Effect Forecasting

Zhixuan Chu, Hui Ding, Guang Zeng, Shiyu Wang, Yiming Li

TL;DR

The paper addresses robust and explainable forecasting under missing data and hidden confounding by formulating a causal interventional forecasting task and proposing CIPS, a system that merges a variational autoencoder with fully conditional specification of multiple imputations to impute missing data, infer latent confounders, and predict post-intervention outcomes. It provides a formal causal framework with a treatment, adjustment, confounders, and outcome, proves identifiability of the interventional distribution under appropriate conditions, and implements an inference/generative model pair that handles incomplete data via FCSMI. Empirical results on real Fintech datasets show CIPS consistently outperforms standard baselines across missingness regimes and demonstrates robustness to data gaps, with ablation confirming the value of multiple imputation. The approach yields a scalable, explainable forecasting pipeline suitable for industrial decision support, particularly in marketing campaign and strategy planning contexts, where external factors and evolving information critically shape outcomes.

Abstract

Although the widespread use of AI systems in today's world is growing, many current AI systems are found vulnerable due to hidden bias and missing information, especially in the most commonly used forecasting system. In this work, we explore the robustness and explainability of AI-based forecasting systems. We provide an in-depth analysis of the underlying causality involved in the effect prediction task and further establish a causal graph based on treatment, adjustment variable, confounder, and outcome. Correspondingly, we design a causal interventional prediction system (CIPS) based on a variational autoencoder and fully conditional specification of multiple imputations. Extensive results demonstrate the superiority of our system over state-of-the-art methods and show remarkable versatility and extensibility in practice.

Causal Interventional Prediction System for Robust and Explainable Effect Forecasting

TL;DR

The paper addresses robust and explainable forecasting under missing data and hidden confounding by formulating a causal interventional forecasting task and proposing CIPS, a system that merges a variational autoencoder with fully conditional specification of multiple imputations to impute missing data, infer latent confounders, and predict post-intervention outcomes. It provides a formal causal framework with a treatment, adjustment, confounders, and outcome, proves identifiability of the interventional distribution under appropriate conditions, and implements an inference/generative model pair that handles incomplete data via FCSMI. Empirical results on real Fintech datasets show CIPS consistently outperforms standard baselines across missingness regimes and demonstrates robustness to data gaps, with ablation confirming the value of multiple imputation. The approach yields a scalable, explainable forecasting pipeline suitable for industrial decision support, particularly in marketing campaign and strategy planning contexts, where external factors and evolving information critically shape outcomes.

Abstract

Although the widespread use of AI systems in today's world is growing, many current AI systems are found vulnerable due to hidden bias and missing information, especially in the most commonly used forecasting system. In this work, we explore the robustness and explainability of AI-based forecasting systems. We provide an in-depth analysis of the underlying causality involved in the effect prediction task and further establish a causal graph based on treatment, adjustment variable, confounder, and outcome. Correspondingly, we design a causal interventional prediction system (CIPS) based on a variational autoencoder and fully conditional specification of multiple imputations. Extensive results demonstrate the superiority of our system over state-of-the-art methods and show remarkable versatility and extensibility in practice.
Paper Structure (19 sections, 1 theorem, 10 equations, 8 figures, 2 tables)

This paper contains 19 sections, 1 theorem, 10 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Structural Causal Model
  4. Variational Autoencoder
  5. Preliminary
  6. Causality Analysis
  7. Clarification on Treatment Effect Estimation
  8. Problem Formulation
  9. Evidence of Causal Relationship in Real Case
  10. Causal Inference Identification
  11. Methodology
  12. Inference model
  13. Generative model
  14. Experiments
  15. Dataset
  16. ...and 4 more sections

Key Result

Theorem 1

If the $p(y| X, M, T= t, Z)$ can be inferred from the observations of $(X,M,T,y)$, then $p\left( y | X, M, do( T= t)\right)$ is identifiable.

Figures (8)

  • Figure 1: The causal graph involved in the CIPS.
  • Figure 2: The causal analysis in the real marketing campaign dataset including (a) $X \rightarrow T$ the distribution of marketing campaigns (treatment) in the customer feature (confounder) space; (b) $X \rightarrow Y$ the distribution of different levels of outcome in the customer feature space (confounder); (c) $T \rightarrow Y$ the distribution of outcomes for different marketing campaigns (treatment); (d) $M \rightarrow Y$ the distribution of outcomes under different external factors (adjustment variable). Axis labels are omitted due to the nonpublic nature of the data.
  • Figure 3: The correlation matrix for all variables in company strategy and marketing campaign datasets. Axis labels are omitted due to the nonpublic nature of the data.
  • Figure 4: Distribution of different treatment assignment (marketing campaign or company strategy) in customer feature (confounder) space. We can observe more distinct bias in the company strategy than in the marketing campaign.
  • Figure 5: Distribution of the different levels of effect outcome in customer feature (confounder) space. We can observe more distinct bias in the company strategy than in the marketing campaign.
  • ...and 3 more figures

Theorems & Definitions (1)

  • Theorem 1