Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Golden Ratio-Based Sufficient Dimension Reduction

Wenjing Yang, Yuhong Yang

TL;DR

This paper introduces GRNN-SDR, a neural-network–based sufficient dimension reduction method that uses a golden-ratio–driven search to identify the structural dimension while accurately estimating the central space $\boldsymbol{S}_{Y|X}$. Grounded in Barron-class function approximation, the method provides theoretical guarantees: an $O(N)$ training complexity for a fixed network and risk bounds that balance approximation and estimation error. The approach accommodates practical dimension reduction via $\delta_N$-approximation and demonstrates strong empirical performance across diverse simulations, often outperforming traditional SDR methods in both accuracy and efficiency. The framework extends naturally to deeper architectures, enabling scalable SDR in high-dimensional settings encountered in modern data applications, with robust performance even when the true dimension is unknown.

Abstract

Many machine learning applications deal with high dimensional data. To make computations feasible and learning more efficient, it is often desirable to reduce the dimensionality of the input variables by finding linear combinations of the predictors that can retain as much original information as possible in the relationship between the response and the original predictors. We propose a neural network based sufficient dimension reduction method that not only identifies the structural dimension effectively, but also estimates the central space well. It takes advantages of approximation capabilities of neural networks for functions in Barron classes and leads to reduced computation cost compared to other dimension reduction methods in the literature. Additionally, the framework can be extended to fit practical dimension reduction, making the methodology more applicable in practical settings.

Golden Ratio-Based Sufficient Dimension Reduction

TL;DR

This paper introduces GRNN-SDR, a neural-network–based sufficient dimension reduction method that uses a golden-ratio–driven search to identify the structural dimension while accurately estimating the central space . Grounded in Barron-class function approximation, the method provides theoretical guarantees: an training complexity for a fixed network and risk bounds that balance approximation and estimation error. The approach accommodates practical dimension reduction via -approximation and demonstrates strong empirical performance across diverse simulations, often outperforming traditional SDR methods in both accuracy and efficiency. The framework extends naturally to deeper architectures, enabling scalable SDR in high-dimensional settings encountered in modern data applications, with robust performance even when the true dimension is unknown.

Abstract

Many machine learning applications deal with high dimensional data. To make computations feasible and learning more efficient, it is often desirable to reduce the dimensionality of the input variables by finding linear combinations of the predictors that can retain as much original information as possible in the relationship between the response and the original predictors. We propose a neural network based sufficient dimension reduction method that not only identifies the structural dimension effectively, but also estimates the central space well. It takes advantages of approximation capabilities of neural networks for functions in Barron classes and leads to reduced computation cost compared to other dimension reduction methods in the literature. Additionally, the framework can be extended to fit practical dimension reduction, making the methodology more applicable in practical settings.

Paper Structure

This paper contains 23 sections, 12 theorems, 61 equations, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Method
  3. Preliminary
  4. Approximation Bound
  5. Error Bound for Training Neural Network
  6. Structural Dimension Analysis
  7. Algorithms
  8. Practical Dimension Reduction
  9. Simulations
  10. Evaluation Metrics
  11. Simulation Study on Noise Level
  12. Simulation Study on Training Size and Computation Time
  13. Simulation Study on Covariate Distribution
  14. Simulation Study on Feature Dimension Size
  15. Simulation Study on Practical Dimension Reduction
  16. ...and 8 more sections

Key Result

Proposition 1

If $f(x) = g(\beta^{T}_dx)$, and $C_g$ is the finite first absolute moment of the Fourier magnitude distribution of $g$. Then there exists a neural network model of two hidden layers, such that where $(\bar{\theta}, \beta_d) \in \Theta_{d,m, \delta, C}$.

Figures (7)

  • Figure 1: Estimation accuracy ($r$) and MSE results for Model 1
  • Figure 2: Estimation accuracy ($r$) and MSE results for Model 2
  • Figure 3: Estimation accuracy ($r$) and computation time for Model 2
  • Figure 4: Estimation accuracy ($r$) and MSE results for Model 3
  • Figure 5: Estimation accuracy ($r$) and MSE results for Model 4
  • ...and 2 more figures

Theorems & Definitions (20)

  • Proposition 1
  • Proposition 2
  • Theorem 1
  • Corollary 1
  • Theorem 2
  • Theorem 3
  • Proposition 3
  • Theorem 4
  • Corollary 2
  • Theorem 5
  • ...and 10 more