Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dynamic Regularized CBDT: Variance-Calibrated Causal Boosting for Interpretable Heterogeneous Treatment Effects

Yichen Liu

TL;DR

We address the challenge of accurately, interpretably, and calibratedly estimating heterogeneous treatment effects from observational data. The Dynamic Regularized CBDT framework extends gradient-boosted decision trees with a composite loss that jointly minimizes prediction error, intra-group variance, global calibration, and ATE calibration, while dynamically updating regularization through gradient statistics. Theoretical guarantees show a PEHE upper-bound reduction of order $O(\sqrt{\lambda+\alpha})$ and convergence properties, complemented by empirical evidence on IHDP and MIMIC-III-based datasets demonstrating improved PEHE, ATE accuracy, calibration coverage, and computational efficiency. The results indicate that dynamic variance regularization yields tighter error bounds and more reliable, interpretable causal rules, with practical impact for high-stakes decision support in healthcare and policy.

Abstract

Heterogeneous treatment effect estimation in high-stakes applications demands models that simultaneously optimize precision, interpretability, and calibration. Many existing tree-based causal inference techniques, however, exhibit high estimation errors when applied to observational data because they struggle to capture complex interactions among factors and rely on static regularization schemes. In this work, we propose Dynamic Regularized Causal Boosted Decision Trees (CBDT), a novel framework that integrates variance regularization and average treatment effect calibration into the loss function of gradient boosted decision trees. Our approach dynamically updates the regularization parameters using gradient statistics to better balance the bias-variance tradeoff. Extensive experiments on standard benchmark datasets and real-world clinical data demonstrate that the proposed method significantly improves estimation accuracy while maintaining reliable coverage of true treatment effects. In an intensive care unit patient triage study, the method successfully identified clinically actionable rules and achieved high accuracy in treatment effect estimation. The results validate that dynamic regularization can effectively tighten error bounds and enhance both predictive performance and model interpretability.

Dynamic Regularized CBDT: Variance-Calibrated Causal Boosting for Interpretable Heterogeneous Treatment Effects

TL;DR

We address the challenge of accurately, interpretably, and calibratedly estimating heterogeneous treatment effects from observational data. The Dynamic Regularized CBDT framework extends gradient-boosted decision trees with a composite loss that jointly minimizes prediction error, intra-group variance, global calibration, and ATE calibration, while dynamically updating regularization through gradient statistics. Theoretical guarantees show a PEHE upper-bound reduction of order and convergence properties, complemented by empirical evidence on IHDP and MIMIC-III-based datasets demonstrating improved PEHE, ATE accuracy, calibration coverage, and computational efficiency. The results indicate that dynamic variance regularization yields tighter error bounds and more reliable, interpretable causal rules, with practical impact for high-stakes decision support in healthcare and policy.

Abstract

Heterogeneous treatment effect estimation in high-stakes applications demands models that simultaneously optimize precision, interpretability, and calibration. Many existing tree-based causal inference techniques, however, exhibit high estimation errors when applied to observational data because they struggle to capture complex interactions among factors and rely on static regularization schemes. In this work, we propose Dynamic Regularized Causal Boosted Decision Trees (CBDT), a novel framework that integrates variance regularization and average treatment effect calibration into the loss function of gradient boosted decision trees. Our approach dynamically updates the regularization parameters using gradient statistics to better balance the bias-variance tradeoff. Extensive experiments on standard benchmark datasets and real-world clinical data demonstrate that the proposed method significantly improves estimation accuracy while maintaining reliable coverage of true treatment effects. In an intensive care unit patient triage study, the method successfully identified clinically actionable rules and achieved high accuracy in treatment effect estimation. The results validate that dynamic regularization can effectively tighten error bounds and enhance both predictive performance and model interpretability.

Paper Structure

This paper contains 47 sections, 4 theorems, 40 equations, 4 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. CBDT Overview and Definition
  5. Composite Loss Function
  6. Group Means.
  7. Composite Loss.
  8. Dynamic Regularization Mechanism
  9. Algorithm Pseudocode
  10. Theoretical Guarantees
  11. Precision in Estimation of Heterogeneous Effects (PEHE)
  12. Summary and Theoretical-Experimental Connection
  13. Theoretical Extensions
  14. Gradient Boosting Tree Formulas
  15. Split Gain.
  16. ...and 32 more sections

Key Result

Theorem 2.1

Under Assumptions 1--5, with $\epsilon_{\text{PEHE}}=\mathbb{E}_X[(\hat{\tau}(X)-\tau(X))^2]$, the incorporation of intra-group variance and ATE calibration reduces the PEHE upper bound by $O\Bigl(\sqrt{\lambda+\alpha}\Bigr)$, where the complexity is measured via Rademacher complexity nie2021.

Figures (4)

  • Figure 1: Dual Y-axis bar chart showing PEHE (log scale) and training time, with an overlay of the Efficiency-Adjusted PEHE.
  • Figure 2: Scatter plot of training time versus PEHE; bubble sizes represent ATE Error, with method labels and color coding for method types.
  • Figure 3: The thermal map shows how each method performs on five key metrics: PEHE, ATE error, training time (seconds), inference time (milliseconds), and efficiency-adjusted PEHE (EAP). The color depth represents the relative value of each indicator, where the darker (or brighter) the area corresponds to the better value (after appropriate normalization and inversion processing).
  • Figure 4: Sensitivity Analysis: Heatmaps for PEHE and ATE error. A divergent blue–white–red color scale is used, with contour lines delineating the safe zone (PEHE $<$ 0.55) and regions of performance collapse indicated.

Theorems & Definitions (4)

  • Theorem 2.1: PEHE Upper Bound Reduction
  • Theorem 2.2: Convergence of Dynamic Regularization
  • Theorem 2.3: Fidelity of Rule Extraction
  • Theorem 3.1: Lemma: Improvement of Estimation Error Upper Bound via Variance Calibration