On the Pointwise Behavior of Recursive Partitioning and Its Implications for Heterogeneous Causal Effect Estimation

TL;DR

This paper shows that adaptive recursive partitioning via decision trees can fail to achieve polynomial convergence rates in the uniform sense over the covariate space, even when pruning is used. In the simple location model, depth-1 trees exhibit end-cut biased splits that yield extremely imbalanced cells and arbitrarily slow pointwise convergence in some regions, and honest trees can be uniformly inconsistent when depth grows as slowly as $K\approx \log\log n$. Remarkably, random forests with subsampling and random feature selection recover near-optimal, uniform pointwise rates, albeit with loss of interpretability and extra tuning parameters. The results have direct implications for heterogeneous causal effect estimation and other pointwise inference tasks, suggesting caution in using CART-type methods in high-stakes, uniform-accuracy scenarios and highlighting the practical value of ensembles for robust performance.

Abstract

Decision tree learning is increasingly being used for pointwise inference. Important applications include causal heterogenous treatment effects and dynamic policy decisions, as well as conditional quantile regression and design of experiments, where tree estimation and inference is conducted at specific values of the covariates. In this paper, we call into question the use of decision trees (trained by adaptive recursive partitioning) for such purposes by demonstrating that they can fail to achieve polynomial rates of convergence in uniform norm with non-vanishing probability, even with pruning. Instead, the convergence may be arbitrarily slow or, in some important special cases, such as honest regression trees, fail completely. We show that random forests can remedy the situation, turning poor performing trees into nearly optimal procedures, at the cost of losing interpretability and introducing two additional tuning parameters. The two hallmarks of random forests, subsampling and the random feature selection mechanism, are seen to each distinctively contribute to achieving nearly optimal performance for the model class considered.

Figures (3)

• Figure 1: Pointwise RMSE of decision stumps for location model.
• Figure 2: Pointwise RMSE of pruned trees for models where $\mathbf{x}$ and $y$ are dependent.
• Figure 3: Pointwise RMSE of random forests for location model.

Theorems & Definitions (23)

• Theorem 4.1
• Remark 1
• Theorem 4.2
• Definition 5.1: Honest CART (CART+)
• Theorem 5.2
• Theorem 7.1
• proof : Proof of \ref{['eq:split_range']} in Theorem \ref{['thm:master']}
• Remark 2
• proof : Proof of \ref{['eq:rate_constant2']} and \ref{['eq:rate_constant']} in Theorem \ref{['thm:rates']}
• proof : Proof of \ref{['eq:honest_lower']}