Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Sufficient Dimension Reduction via Hellinger Correlation

Seungbeom Hong, Ilmun Kim, Jun Song

TL;DR

This paper addresses sufficient dimension reduction for single-index models by exploiting the Hellinger correlation as a dependence-based objective. By maximizing $H(\alpha^\top X, Y)$ over the unit sphere, it identifies the central subspace $\mathrm{Span}(\eta_0)$ with consistency guarantees, without requiring strong independence assumptions. The approach is implemented via a two-stage optimization that leverages initial values from standard SDR methods and refined by a downhill simplex search, with complexity $O(n^2 p k)$ per iteration. Empirical results across simulated and real data demonstrate substantial improvements over traditional SDR methods and related dependence-based approaches, underscoring the method's robustness to nonlinearity and heavy tails. The work offers a principled, theoretically justified tool for dimension reduction in high-dimensional supervised learning contexts and lays groundwork for extensions to multi-index models and classification.

Abstract

In this work, we develop a new theory and method for sufficient dimension reduction (SDR) in single-index models, where SDR is a sub-field of supervised dimension reduction based on conditional independence. Our work is primarily motivated by the recent introduction of the Hellinger correlation as a dependency measure. Utilizing this measure, we develop a method capable of effectively detecting the dimension reduction subspace, complete with theoretical justification. Through extensive numerical experiments, we demonstrate that our proposed method significantly enhances and outperforms existing SDR methods. This improvement is largely attributed to our proposed method's deeper understanding of data dependencies and the refinement of existing SDR techniques.

Enhancing Sufficient Dimension Reduction via Hellinger Correlation

TL;DR

This paper addresses sufficient dimension reduction for single-index models by exploiting the Hellinger correlation as a dependence-based objective. By maximizing over the unit sphere, it identifies the central subspace with consistency guarantees, without requiring strong independence assumptions. The approach is implemented via a two-stage optimization that leverages initial values from standard SDR methods and refined by a downhill simplex search, with complexity per iteration. Empirical results across simulated and real data demonstrate substantial improvements over traditional SDR methods and related dependence-based approaches, underscoring the method's robustness to nonlinearity and heavy tails. The work offers a principled, theoretically justified tool for dimension reduction in high-dimensional supervised learning contexts and lays groundwork for extensions to multi-index models and classification.

Abstract

In this work, we develop a new theory and method for sufficient dimension reduction (SDR) in single-index models, where SDR is a sub-field of supervised dimension reduction based on conditional independence. Our work is primarily motivated by the recent introduction of the Hellinger correlation as a dependency measure. Utilizing this measure, we develop a method capable of effectively detecting the dimension reduction subspace, complete with theoretical justification. Through extensive numerical experiments, we demonstrate that our proposed method significantly enhances and outperforms existing SDR methods. This improvement is largely attributed to our proposed method's deeper understanding of data dependencies and the refinement of existing SDR techniques.
Paper Structure (13 sections, 4 theorems, 40 equations, 1 figure, 13 tables)

This paper contains 13 sections, 4 theorems, 40 equations, 1 figure, 13 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. SDR through dependence measure
  4. Copula
  5. f-divergence
  6. Hellinger Correlation
  7. Main Results
  8. Method
  9. Theoretical Results
  10. Numerical Experiments
  11. Real Data Analysis
  12. Discussion
  13. Additional Simulation Results

Key Result

Theorem 2.1

Let ${X} = (X_1, X_2,\dotsc, X_p)^{\hbox{\tiny{\sf T}}}$ be a random vector. Suppose that $F$ and $f$ are the joint cumulative distribution function and the joint probability density function of ${X}$. Then, there exists a function $C : [0, 1]^p \to [0, 1]$, called the copula of ${X}$, such that Additionally, there exists a function $c : [0, 1]^{p} \to [0, \infty)$, called the copula density of $

Figures (1)

  • Figure 1: Boxplots of $\Delta(\mathcal{S}_{True}, \mathcal{S}_{Estimated})$ over 100 samples of size $n=100$ with normal predictors. We compare the performance between SDR Method (aqua blue) and our proposed SDR Method-HC (light coral). As shown, our proposed method consistently outperforms the corresponding SDR methods.

Theorems & Definitions (8)

  • Theorem 2.1: Sklar, 1959
  • Definition 2.2
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • proof