Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A simplified directional KeRF algorithm

Iakovidis Isidoros, Nicola Arcozzi

TL;DR

The paper addresses kernel-based random forests (KeRF) and their interpolation properties, introducing a simplified directional KeRF to reduce data-dependent splitting. It proves that the simplified directional KeRF is asymptotically equivalent to the centered KeRF as the number of trees $M$ tends to infinity, and it derives a kernel representation for the estimator. It provides improved convergence rates in the mean interpolation regime for the centered KeRF and demonstrates, via numerical experiments, that the two approaches behave identically for moderate to large $M$. The results offer a simpler, kernel-based alternative with theoretical guarantees and practical viability in low-dimensional settings.

Abstract

Random forest methods belong to the class of non-parametric machine learning algorithms. They were first introduced in 2001 by Breiman and they perform with accuracy in high dimensional settings. In this article, we consider, a simplified kernel-based random forest algorithm called simplified directional KeRF (Kernel Random Forest). We establish the asymptotic equivalence between simplified directional KeRF and centered KeRF, with additional numerical experiments supporting our theoretical results.

A simplified directional KeRF algorithm

TL;DR

The paper addresses kernel-based random forests (KeRF) and their interpolation properties, introducing a simplified directional KeRF to reduce data-dependent splitting. It proves that the simplified directional KeRF is asymptotically equivalent to the centered KeRF as the number of trees tends to infinity, and it derives a kernel representation for the estimator. It provides improved convergence rates in the mean interpolation regime for the centered KeRF and demonstrates, via numerical experiments, that the two approaches behave identically for moderate to large . The results offer a simpler, kernel-based alternative with theoretical guarantees and practical viability in low-dimensional settings.

Abstract

Random forest methods belong to the class of non-parametric machine learning algorithms. They were first introduced in 2001 by Breiman and they perform with accuracy in high dimensional settings. In this article, we consider, a simplified kernel-based random forest algorithm called simplified directional KeRF (Kernel Random Forest). We establish the asymptotic equivalence between simplified directional KeRF and centered KeRF, with additional numerical experiments supporting our theoretical results.
Paper Structure (10 sections, 14 theorems, 60 equations, 7 figures)

This paper contains 10 sections, 14 theorems, 60 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Contributions of the article
  3. Notation and preliminaries
  4. Kernel Random Forest algorithm
  5. The Centered KeRF algorithm
  6. Simplified directional KeRF
  7. Plots and experiments.
  8. Interpolating random trees
  9. Conclusions
  10. Proofs of theorems

Key Result

Theorem 1

$\textbf{Y}=m(\textbf{X}) +\epsilon$ where $\epsilon$ is a zero mean Gaussian noise with finite variance $\sigma$ independent of $\textbf{X}$. Assuming also that $\textbf{X}$ is uniformly distributed in $[0,1]^d$ and $m$ is a Lipschitz function. Then there exists constants $c_1, c_2$ depending on $d

Figures (7)

  • Figure 1: Centered algorithm with tree level $k=1$ with the convention that $1$ corresponds to $x_1$ axis and $2$ to the $x_2$ axis.
  • Figure 2: Centered algorithm with tree level $k=2$ with the convention that $1$ corresponds to $x_1$ axis and $2$ to the $x_2$ axis.
  • Figure 3: Centered algorithm with tree level $k=1$ with the convention that $1$ corresponds to $x_1$ axis and $2$ to the $x_2$ axis.
  • Figure 4: Centered algorithm with tree level $k=2$ with the convention that $1$ corresponds to $x_1$ axis and $2$ to the $x_2$ axis.
  • Figure 5: Comparison of $L-$error and standard deviation.
  • ...and 2 more figures

Theorems & Definitions (24)

  • Theorem
  • Definition 1
  • Definition 2
  • Proposition 1: Scornet S, Proposition 1
  • Definition 3
  • Proposition 2: Scornet S, Proposition 2
  • Proposition 3
  • Theorem 1
  • Corollary 1
  • Definition 4
  • ...and 14 more