Skeleton Regression: A Graph-Based Approach to Estimation with Manifold Structure

TL;DR

This work introduces a graph-based skeleton regression framework for covariates concentrated around low-dimensional manifolds, projecting data onto a learned skeleton and applying graph-aware nonparametric regression. It develops three methods—S-Kernel, S-kNN, and S-Lspline—along with consistency and convergence results for edge and knot points, and demonstrates robustness to noise and improved accuracy on simulated and real datasets. The approach mitigates the curse of dimensionality by operating in the intrinsic, graph-defined space and supports interpretable visualization of manifold structure. Together, the theoretical guarantees and extensive simulations/real-data experiments underscore the practicality and scalability of skeleton-based regression for geometry-driven data analysis.

Abstract

We introduce a new regression framework designed to deal with large-scale, complex data that lies around a low-dimensional manifold with noises. Our approach first constructs a graph representation, referred to as the skeleton, to capture the underlying geometric structure. We then define metrics on the skeleton graph and apply nonparametric regression techniques, along with feature transformations based on the graph, to estimate the regression function. We also discuss the limitations of some nonparametric regressors with respect to the general metric space such as the skeleton graph. The proposed regression framework suggests a novel way to deal with data with underlying geometric structures and provides additional advantages in handling the union of multiple manifolds, additive noises, and noisy observations. We provide statistical guarantees for the proposed method and demonstrate its effectiveness through simulations and real data examples.

Figures (18)

• Figure 1: Skeleton Regression illustrated by data with covariates having the shape of two moons in a 2D space.
• Figure 2: Orange shaded area illustrates the 2-NN region between knots $1$ and $2$.
• Figure 3: Illustration of skeleton-based distance. Let $C_1, C_2, C_3, C_4$ be the knots, and let $S_2,S_3,S_4$ be the mid-point on the edges $E_{12},E_{23},E_{34}$ respectively. Let $S_1$ bet the midpoint between $C_1$ and $S_2$ on the edge. Let $d_{ij} = \left\Vert C_i - C_j\right\Vert$ denotes the length of the edge $E_{ij}$. $d_{\mathcal{S}}(S_1,S_2) = \frac{1}{4} d_{12}$ illustrated by the blue path. $d_{\mathcal{S}}(S_2,S_3) = \frac{1}{2} d_{12} + \frac{1}{2} d_{23}$ illustrated by the green path. $d_{\mathcal{S}}(S_2,S_4) = \frac{1}{2} d_{12} + d_{23} + \frac{1}{2} d_{34}$ illustrated by the orange path.
• Figure 4: Illustration of projection to the skeleton. The skeleton structure is given by the black dots and lines. Data point $X_1$ is projected to $S_1$ on the edge between $C_1$ and $C_2$. Data point $X_2$ is projected to knot $C_2$.
• Figure 5: Yinyang Regression Data

Theorems & Definitions (14)

• Remark 1
• Remark 2
• Remark 3
• Theorem 1: Consistency on Edge Points
• Theorem 2: Consistency on Knots with Nonzero Mass
• Proposition 3
• Remark 4
• Remark 5
• Theorem 3.1: Linear spline representer theorem
• proof