Adaptive Estimation of Multivariate Piecewise Polynomials and Bounded Variation Functions by Optimal Decision Trees

Abstract

Proposed by Donoho (1997), Dyadic CART is a nonparametric regression method which computes a globally optimal dyadic decision tree and fits piecewise constant functions in two dimensions. In this article we define and study Dyadic CART and a closely related estimator, namely Optimal Regression Tree (ORT), in the context of estimating piecewise smooth functions in general dimensions in the fixed design setup. More precisely, these optimal decision tree estimators fit piecewise polynomials of any given degree. Like Dyadic CART in two dimensions, we reason that these estimators can also be computed in polynomial time in the sample size $N$ via dynamic programming. We prove oracle inequalities for the finite sample risk of Dyadic CART and ORT which imply tight risk bounds for several function classes of interest. Firstly, they imply that the finite sample risk of ORT of order $r \geq 0$ is always bounded by $C k \frac{\log N}{N}$ whenever the regression function is piecewise polynomial of degree $r$ on some reasonably regular axis aligned rectangular partition of the domain with at most $k$ rectangles. Beyond the univariate case, such guarantees are scarcely available in the literature for computationally efficient estimators. Secondly, our oracle inequalities uncover minimax rate optimality and adaptivity of the Dyadic CART estimator for function spaces with bounded variation. We consider two function spaces of recent interest where multivariate total variation denoising and univariate trend filtering are the state of the art methods. We show that Dyadic CART enjoys certain advantages over these estimators while still maintaining all their known guarantees.

Figures (5)

• Figure 1: Figure (a) is an example of a recursive dyadic partition of the square. Figure (b) is nondyadic but is a hierarchical partition. Figure (c) is an example of a nonhierarchical partition. An easy way to see this is that there is no split from top to bottom or left to right.
• Figure 2: Figure depicts the ground truth matrix which is piecewise constant on a non hierarchical partition.
• Figure 3: This is a $\log$ MSE vs $\log N$ plot for ORT (in green) and ideal TVD (in blue). The slope for ideal TVD comes out to be $-0.67$ and and the slope for ORT comes out to be $-0.9$
• Figure 4: The two figures are $\log$ MSE vs $\log N$ plot for ideal TVD (in blue) and Dyadic CART (in green). For the figure on the left, the ground truth is piecewise constant with two pieces. The slopes came out to be $-0.71$ and $-1.23$ for ideal TVD and Dyadic CART respectively. For the figure on the right, the ground truth is a smooth bump function. The slopes came out to be $-0.55$ and $-0.56$ for ideal TVD and Dyadic CART respectively.
• Figure 5: The figure on the left is $\log$ MSE vs $\log N$ (base $2$) plot for ideal Trend Filtering (in blue) and Dyadic CART (in green). The slopes came out to be $-0.65$ and $-0.70$ for ideal Trend Filtering and Dyadic Cart respectively. Figure on the right is an instance of our simulation with sample size $N = 256.$ The red piecewise linear curve is the ground truth. The green curve is the ideal Trend Filtering fit and the orange curve is the Dyadic CART fit with $\lambda = 8$.

Theorems & Definitions (69)

• lemma 1
• remark 1
• remark 2
• theorem \oldthetheorem
• lemma 2
• remark 3
• remark 4
• corollary 1
• proposition 1: berman2002exact
• theorem \oldthetheorem