Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inference with Mondrian Random Forests

Matias D. Cattaneo, Jason M. Klusowski, William G. Underwood

TL;DR

This work analyzes Mondrian random forests as a purely random partitioning approach to nonparametric regression and develops rigorous statistical inference for the regression function μ. It proves a Berry–Esseen–type central limit theorem for the Mondrian RF estimator and builds a feasible variance estimator to enable valid confidence intervals without data splitting. To further enhance estimation, the authors introduce a debiasing framework based on generalized jackknife across multiple lifetimes λ, achieving minimax optimal pointwise rates for β-Hölder functions and yielding a CLT and valid CIs in the debiased setting. The paper also provides practical guidance for tuning λ and B, proposes online and batch algorithms with complexity guarantees, and validates the theory through simulations showing strong finite-sample performance. Collectively, the results advance inference with Mondrian forests and offer practical tools for uncertainty quantification in nonparametric regression.

Abstract

Random forests are popular methods for regression and classification analysis, and many different variants have been proposed in recent years. One interesting example is the Mondrian random forest, in which the underlying constituent trees are constructed via a Mondrian process. We give precise bias and variance characterizations, along with a Berry-Esseen-type central limit theorem, for the Mondrian random forest regression estimator. By combining these results with a carefully crafted debiasing approach and an accurate variance estimator, we present valid statistical inference methods for the unknown regression function. These methods come with explicitly characterized error bounds in terms of the sample size, tree complexity parameter, and number of trees in the forest, and include coverage error rates for feasible confidence interval estimators. Our debiasing procedure for the Mondrian random forest also allows it to achieve the minimax-optimal point estimation convergence rate in mean squared error for multivariate $β$-Hölder regression functions, for all $β> 0$, provided that the underlying tuning parameters are chosen appropriately. Efficient and implementable algorithms are devised for both batch and online learning settings, and we study the computational complexity of different Mondrian random forest implementations. Finally, simulations with synthetic data validate our theory and methodology, demonstrating their excellent finite-sample properties.

Inference with Mondrian Random Forests

TL;DR

This work analyzes Mondrian random forests as a purely random partitioning approach to nonparametric regression and develops rigorous statistical inference for the regression function μ. It proves a Berry–Esseen–type central limit theorem for the Mondrian RF estimator and builds a feasible variance estimator to enable valid confidence intervals without data splitting. To further enhance estimation, the authors introduce a debiasing framework based on generalized jackknife across multiple lifetimes λ, achieving minimax optimal pointwise rates for β-Hölder functions and yielding a CLT and valid CIs in the debiased setting. The paper also provides practical guidance for tuning λ and B, proposes online and batch algorithms with complexity guarantees, and validates the theory through simulations showing strong finite-sample performance. Collectively, the results advance inference with Mondrian forests and offer practical tools for uncertainty quantification in nonparametric regression.

Abstract

Random forests are popular methods for regression and classification analysis, and many different variants have been proposed in recent years. One interesting example is the Mondrian random forest, in which the underlying constituent trees are constructed via a Mondrian process. We give precise bias and variance characterizations, along with a Berry-Esseen-type central limit theorem, for the Mondrian random forest regression estimator. By combining these results with a carefully crafted debiasing approach and an accurate variance estimator, we present valid statistical inference methods for the unknown regression function. These methods come with explicitly characterized error bounds in terms of the sample size, tree complexity parameter, and number of trees in the forest, and include coverage error rates for feasible confidence interval estimators. Our debiasing procedure for the Mondrian random forest also allows it to achieve the minimax-optimal point estimation convergence rate in mean squared error for multivariate -Hölder regression functions, for all , provided that the underlying tuning parameters are chosen appropriately. Efficient and implementable algorithms are devised for both batch and online learning settings, and we study the computational complexity of different Mondrian random forest implementations. Finally, simulations with synthetic data validate our theory and methodology, demonstrating their excellent finite-sample properties.
Paper Structure (47 sections, 21 theorems, 227 equations, 1 figure, 6 tables, 2 algorithms)

This paper contains 47 sections, 21 theorems, 227 equations, 1 figure, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Organization
  4. Notation
  5. Setup
  6. Data generation
  7. The Mondrian process
  8. Mondrian random forests
  9. Inference with Mondrian random forests
  10. Bias and variance characterizations
  11. Central limit theorem
  12. Confidence intervals
  13. Debiased Mondrian random forests
  14. Bias and variance characterizations
  15. Minimax optimality
  16. ...and 32 more sections

Key Result

Lemma 1

Suppose Assumptions ass:data and ass:estimator hold. For each $1 \leq r \leq \lfloor {\ThisStyle{ \ensurestackMath{\stackengine{-0.5\LMpt}{\SavedStyle \beta} {\SavedStyle {}} {U}{c}{F}{F}{S}}\newline}} / 2 \rfloor$ there exists $B_r(x) \in \mathbb{R}$, which is a function of the derivati Whenever $\beta > 2$, the leading bias is the quadratic term If $X_i \sim \mathop{\mathrm{Unif}}\n

Figures (1)

  • Figure 1: The Mondrian process $T \sim {\mathcal{M}} ( [0,1]^d, \lambda )$ with $d=2$ and lifetime parameters $\lambda$.

Theorems & Definitions (43)

  • Definition 1: Hölder continuity
  • Lemma 1: Bias
  • Lemma 2: Variance
  • Theorem 1: Mean squared error
  • Theorem 2: Central limit theorem
  • Lemma 3: Variance estimation
  • Theorem 3: Confidence intervals
  • Lemma 4: Bias of the debiased estimator
  • Lemma 5: Variance of the debiased estimator
  • Theorem 4: Mean squared error of the debiased estimator
  • ...and 33 more