Random Forest Weighted Local Fréchet Regression with Random Objects

TL;DR

This work introduces two random forest–weighted Fréchet regression methods for metric-space responses: the local constant (RFWLCFR) and local linear (RFWLLFR) variants. By replacing traditional kernel weights with a locally adaptive forest kernel, the approach mitigates the curse of dimensionality and achieves strong theoretical guarantees via infinite order U-statistics, including consistency, convergence rates, and asymptotic normality for the local constant estimator. Extensive simulations across distributions, SPD matrices, and sphere data, plus real-data applications to New York taxi networks and mortality histograms, demonstrate substantial predictive gains and meaningful variable importance insights. The framework unifies nonparametric Fréchet regression with random forests, broadening applicability to complex responses while preserving statistical rigor and interpretability.

Abstract

Statistical analysis is increasingly confronted with complex data from metric spaces. Petersen and Müller (2019) established a general paradigm of Fréchet regression with complex metric space valued responses and Euclidean predictors. However, the local approach therein involves nonparametric kernel smoothing and suffers from the curse of dimensionality. To address this issue, we in this paper propose a novel random forest weighted local Fréchet regression paradigm. The main mechanism of our approach relies on a locally adaptive kernel generated by random forests. Our first method uses these weights as the local average to solve the conditional Fréchet mean, while the second method performs local linear Fréchet regression, both significantly improving existing Fréchet regression methods. Based on the theory of infinite order U-processes and infinite order $M_{m_n}$-estimator, we establish the consistency, rate of convergence, and asymptotic normality for our local constant estimator, which covers the current large sample theory of random forests with Euclidean responses as a special case. Numerical studies show the superiority of our methods with several commonly encountered types of responses such as distribution functions, symmetric positive-definite matrices, and sphere data. The practical merits of our proposals are also demonstrated through the application to New York taxi data and human mortality data.

Figures (13)

• Figure 1: The first plot illustrates the flow statistics of yellow taxis in ten distinct zones of Manhattan, New York, during a certain time period. The thickness of the edges connecting vertices corresponds to the level of inter-zone traffic, while the size of vertices represents the total traffic volume within each zone. The remaining five plots from left to right are the predictions given by the global Fréchet regression, local Fréchet regression after dimension reduction, single index Fréchet regression, RFWLCFR and RFWLLFR.
• Figure 2: Weights given by the random forest kernel. Each point represents a training sample. The red points represent samples whose weights to $(0.5,0.5)$ are greater than $0$ and the diameter of these points indicates the size of the weights.
• Figure 3: The relationships among the eight local estimators.
• Figure 4: The variable importance for all variables of setting I-1.
• Figure 5: The variable importance for all variables of setting II-1.

Theorems & Definitions (19)

• Example 1
• Theorem 1
• Remark 2
• Theorem 3
• Lemma 4
• Remark 5
• Lemma 6
• Remark 7
• Theorem 8
• Theorem 9