Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Machine Learning for Stress Testing: Uncertainty Decomposition in Causal Panel Prediction

Yu Wang, Xiangchen Liu, Siguang Li

TL;DR

This work proposes a framework for policy-path counterfactual inference in panels that transparently separates what can be learned from data from what requires assumptions about confounding, and produces a three-layer uncertainty decomposition that cleanly separates estimation uncertainty from confounding uncertainty.

Abstract

Regulatory stress testing requires projecting credit losses under hypothetical macroeconomic scenarios -- a fundamentally causal question typically treated as a prediction problem. We propose a framework for policy-path counterfactual inference in panels that transparently separates what can be learned from data from what requires assumptions about confounding. Our approach has four components: (i) observational identification of path-conditional means via iterated regression, enabling continuous macro-path contrasts without requiring a control group; (ii) causal set identification under bounded confounding, yielding sharp identified sets with interpretable breakdown values that communicate robustness in a single number; (iii) an oracle inequality showing that recursive rollout error is governed by a horizon-dependent amplification factor, providing a concrete answer to how far ahead one can reliably predict under stress; and (iv) importance-weighted conformal calibration bands with diagnostics that quantify extrapolation cost and trigger abstention when coverage guarantees degrade. The final output is a three-layer uncertainty decomposition that cleanly separates estimation uncertainty from confounding uncertainty. We validate all results through simulation and semi-synthetic experiments with real unemployment data, including a Covid retrospective demonstrating the framework's diagnostic value under extreme scenarios.

Machine Learning for Stress Testing: Uncertainty Decomposition in Causal Panel Prediction

TL;DR

This work proposes a framework for policy-path counterfactual inference in panels that transparently separates what can be learned from data from what requires assumptions about confounding, and produces a three-layer uncertainty decomposition that cleanly separates estimation uncertainty from confounding uncertainty.

Abstract

Regulatory stress testing requires projecting credit losses under hypothetical macroeconomic scenarios -- a fundamentally causal question typically treated as a prediction problem. We propose a framework for policy-path counterfactual inference in panels that transparently separates what can be learned from data from what requires assumptions about confounding. Our approach has four components: (i) observational identification of path-conditional means via iterated regression, enabling continuous macro-path contrasts without requiring a control group; (ii) causal set identification under bounded confounding, yielding sharp identified sets with interpretable breakdown values that communicate robustness in a single number; (iii) an oracle inequality showing that recursive rollout error is governed by a horizon-dependent amplification factor, providing a concrete answer to how far ahead one can reliably predict under stress; and (iv) importance-weighted conformal calibration bands with diagnostics that quantify extrapolation cost and trigger abstention when coverage guarantees degrade. The final output is a three-layer uncertainty decomposition that cleanly separates estimation uncertainty from confounding uncertainty. We validate all results through simulation and semi-synthetic experiments with real unemployment data, including a Covid retrospective demonstrating the framework's diagnostic value under extreme scenarios.
Paper Structure (61 sections, 9 theorems, 11 equations, 4 figures, 2 tables)

This paper contains 61 sections, 9 theorems, 11 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Practical relevance.
  4. Related Work
  5. Causal panel methods.
  6. Sensitivity analysis and partial identification.
  7. Conformal prediction.
  8. Machine learning for stress testing.
  9. Framework: Partial Identification of Policy Paths
  10. Setup and Estimands
  11. Causal Set Identification
  12. Non-Asymptotic Error Bounds
  13. Calibration Bands and Extrapolation Diagnostics
  14. Three-Layer Uncertainty Decomposition
  15. Experiments
  16. ...and 46 more sections

Key Result

Proposition 1

Under Assumptions 1--4, the observational conditional mean $\mu^{\mathrm{obs}}_h(a)$ is identified from pre-period data for all $h=1,\dots,H$, by iterating the identified one-step transition kernel starting from the observed state $I_{i,t_0}$.

Figures (4)

  • Figure 1: Experiment 1A: Oracle inequality validation. Red: recursive error. Black dashed: bound $\epsilon_n \Gamma_h$. Blue: direct estimator. The bound holds in all three regimes; crossover at $h^* \approx 6$ in the near-critical case.
  • Figure 2: Experiment 1B: Mean-state bias $|b_h|$. Linear DGP (blue) $\approx 0$; nonlinear DGP (orange) grows with horizon.
  • Figure 3: Experiment 1D. (a) Identified set at $h=12$ across confounding levels. (b) Three-layer uncertainty: point estimate (black), calibration band (blue), confounding envelopes (yellow--orange), true $\tau^{\mathrm{do}}_h$ (red dashed). (c) Breakdown frontier.
  • Figure 4: Layer 2: Semi-synthetic experiments with real FRED unemployment. Row 1: Oracle inequality (2A). Row 2: Mean-state bias (2B), calibration coverage and $B_{\mathrm{eff}}$ (2C). Row 3: Confounding gap (2D), three-layer uncertainty (2D), COVID retrospective (2E).

Theorems & Definitions (24)

  • Definition 1: Path contrasts
  • Proposition 1: Observational identification
  • Theorem 1: Causal set identification
  • proof : Proof sketch
  • Theorem 2: Oracle inequality for recursive rollout
  • proof : Proof sketch
  • Proposition 2: Weighted calibration coverage
  • Corollary 1: Three-layer uncertainty
  • Definition 2: Cumulative path effect
  • proof : Proof of Proposition \ref{['prop:obs_id']}
  • ...and 14 more