Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformal Prediction for Causal Effects of Continuous Treatments

Maresa Schröder, Dennis Frauen, Jonas Schweisthal, Konstantin Heß, Valentyn Melnychuk, Stefan Feuerriegel

TL;DR

This work is the first to propose conformal prediction for continuous treatments when the propensity score is unknown and must be estimated from data, and derives finite-sample prediction intervals for potential outcomes of continuous treatments.

Abstract

Uncertainty quantification of causal effects is crucial for safety-critical applications such as personalized medicine. A powerful approach for this is conformal prediction, which has several practical benefits due to model-agnostic finite-sample guarantees. Yet, existing methods for conformal prediction of causal effects are limited to binary/discrete treatments and make highly restrictive assumptions such as known propensity scores. In this work, we provide a novel conformal prediction method for potential outcomes of continuous treatments. We account for the additional uncertainty introduced through propensity estimation so that our conformal prediction intervals are valid even if the propensity score is unknown. Our contributions are three-fold: (1) We derive finite-sample prediction intervals for potential outcomes of continuous treatments. (2) We provide an algorithm for calculating the derived intervals. (3) We demonstrate the effectiveness of the conformal prediction intervals in experiments on synthetic and real-world datasets. To the best of our knowledge, we are the first to propose conformal prediction for continuous treatments when the propensity score is unknown and must be estimated from data.

Conformal Prediction for Causal Effects of Continuous Treatments

TL;DR

This work is the first to propose conformal prediction for continuous treatments when the propensity score is unknown and must be estimated from data, and derives finite-sample prediction intervals for potential outcomes of continuous treatments.

Abstract

Uncertainty quantification of causal effects is crucial for safety-critical applications such as personalized medicine. A powerful approach for this is conformal prediction, which has several practical benefits due to model-agnostic finite-sample guarantees. Yet, existing methods for conformal prediction of causal effects are limited to binary/discrete treatments and make highly restrictive assumptions such as known propensity scores. In this work, we provide a novel conformal prediction method for potential outcomes of continuous treatments. We account for the additional uncertainty introduced through propensity estimation so that our conformal prediction intervals are valid even if the propensity score is unknown. Our contributions are three-fold: (1) We derive finite-sample prediction intervals for potential outcomes of continuous treatments. (2) We provide an algorithm for calculating the derived intervals. (3) We demonstrate the effectiveness of the conformal prediction intervals in experiments on synthetic and real-world datasets. To the best of our knowledge, we are the first to propose conformal prediction for continuous treatments when the propensity score is unknown and must be estimated from data.
Paper Structure (42 sections, 10 theorems, 82 equations, 12 figures, 5 tables, 2 algorithms)

This paper contains 42 sections, 10 theorems, 82 equations, 12 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem formulation
  4. CP intervals for potential outcomes of continuous treatments
  5. Scenario 1: Known propensity score
  6. Scenario 2: Unknown treatment policy
  7. Experiments
  8. Datasets
  9. Results for synthetic datasets
  10. Results for the MIMIC dataset
  11. Discussion
  12. Additional theoretical results
  13. Calculating prediction intervals for further causal quantities and differences
  14. Alternative scenario 2: Fixing an approximation of $\delta_{a^{\ast}}(a)$
  15. Soft-interventions on estimated propensities
  16. ...and 27 more sections

Key Result

Lemma 4.1

Let $\mathcal{F}$ define a finite-dimensional function class that includes the function $f$ characterizing the shift in the (potentially unknown) propensity function $\pi$ (see Eq. eqn:prop_shift). Define the distribution-shift-calibrated $(1-\alpha)$-quantile of the non-conformity scores as for an imputed guess $S$ of the $(n+1)$-th non-conformity score $S_{n+1}$. The prediction interval for th

Figures (12)

  • Figure 1: CP intervals on finite-sample data. UQ methods with asymptotic guarantees might suffer from under-coverage and are often not faithful. Thus, we aim at CP with finite-sample guarantees.
  • Figure 2: Key works on causal CP.
  • Figure 3: Use cases of the two scenarios: The new assignment is a function of the original policy (i.e., soft intervention). The policy in the dataset is unknown. The new assignment cannot be expressed as a function of the original policy (i.e., hard intervention).
  • Figure 4: Comparison of faithfulness on dataset 1 across 50 runs. Larger values are better. For each $\alpha$, the plots show how often the empirical intervals contain the true outcome. Intervals should ideally yield a coverage of $1-\alpha$ (red line).
  • Figure 5: Comparison of faithfulness on dataset 2 across 50 runs. Larger values are better.
  • ...and 7 more figures

Theorems & Definitions (17)

  • Lemma 4.1: Gibbs.2023
  • Theorem 4.2: Conformal prediction intervals for known baseline policy
  • proof
  • Lemma 4.3
  • proof
  • Lemma 4.4
  • Theorem 4.5: Conformal prediction intervals for unknown propensity scores
  • proof
  • Lemma A.1
  • proof
  • ...and 7 more