Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Support estimation in high-dimensional heteroscedastic mean regression

Philipp Hermann, Hajo Holzmann

TL;DR

This work addresses support estimation in high-dimensional linear mean regression under random design with heteroscedastic, heavy-tailed errors by proposing a weighted adaptive LASSO estimator built on a smooth pseudo Huber loss. The authors prove sign-consistency and optimal $\ell_\infty$-convergence rates under mild moment conditions, carefully handling the potential mismatch between the robustified and true supports via a primal-dual witness analysis. The theory accommodates two routes depending on the initial estimator’s accuracy ($\ell_\infty$ vs $\ell_2/\ell_1$ bounds) and prescribes scaling $\alpha_n \asymp (\log p / n)^{1/2}$ and $\lambda_n \asymp (\log p)/n$, with a beta-min condition ensuring exact support recovery. Empirical results, including simulations with heavy tails and heteroscedasticity and a riboflavin real-data example, show the proposed method is robust and competitive, especially when combined with knockoffs for false discovery rate control.

Abstract

A current strand of research in high-dimensional statistics deals with robustifying the available methodology with respect to deviations from the pervasive light-tail assumptions. In this paper we consider a linear mean regression model with random design and potentially heteroscedastic, heavy-tailed errors, and investigate support estimation in this framework. We use a strictly convex, smooth variant of the Huber loss function with tuning parameter depending on the parameters of the problem, as well as the adaptive LASSO penalty for computational efficiency. For the resulting estimator we show sign-consistency and optimal rates of convergence in the $\ell_\infty$ norm as in the homoscedastic, light-tailed setting. In our analysis, we have to deal with the issue that the support of the target parameter in the linear mean regression model and its robustified version may differ substantially even for small values of the tuning parameter of the Huber loss function. Simulations illustrate the favorable numerical performance of the proposed methodology.

Support estimation in high-dimensional heteroscedastic mean regression

TL;DR

This work addresses support estimation in high-dimensional linear mean regression under random design with heteroscedastic, heavy-tailed errors by proposing a weighted adaptive LASSO estimator built on a smooth pseudo Huber loss. The authors prove sign-consistency and optimal -convergence rates under mild moment conditions, carefully handling the potential mismatch between the robustified and true supports via a primal-dual witness analysis. The theory accommodates two routes depending on the initial estimator’s accuracy ( vs bounds) and prescribes scaling and , with a beta-min condition ensuring exact support recovery. Empirical results, including simulations with heavy tails and heteroscedasticity and a riboflavin real-data example, show the proposed method is robust and competitive, especially when combined with knockoffs for false discovery rate control.

Abstract

A current strand of research in high-dimensional statistics deals with robustifying the available methodology with respect to deviations from the pervasive light-tail assumptions. In this paper we consider a linear mean regression model with random design and potentially heteroscedastic, heavy-tailed errors, and investigate support estimation in this framework. We use a strictly convex, smooth variant of the Huber loss function with tuning parameter depending on the parameters of the problem, as well as the adaptive LASSO penalty for computational efficiency. For the resulting estimator we show sign-consistency and optimal rates of convergence in the norm as in the homoscedastic, light-tailed setting. In our analysis, we have to deal with the issue that the support of the target parameter in the linear mean regression model and its robustified version may differ substantially even for small values of the tuning parameter of the Huber loss function. Simulations illustrate the favorable numerical performance of the proposed methodology.

Paper Structure

This paper contains 27 sections, 20 theorems, 255 equations, 3 figures, 23 tables.

Table of Contents

  1. Introduction
  2. The adaptive LASSO with pseudo Huber loss function
  3. Sign-consistency and rate of convergence in $\ell_\infty$ norm
  4. Simulations
  5. Effects of tails and heteroscedasticity
  6. Incorporating knockoffs
  7. Real data example
  8. Conclusions
  9. Proofs: Main steps
  10. Outline of the steps of the proof
  11. Preparations
  12. Reduction of estimator to compact domain
  13. Bounding the approximation error
  14. Restricted strong convexity and properties of derivatives
  15. Primal-dual witness approach
  16. ...and 12 more sections

Key Result

Theorem 1

In model generalregressionmodel under Assumption assfan2017, consider the adaptive LASSO estimator $\widehat{\beta}_n^{\,{\mathrm{ALPH}}}$ with initial estimator $\widehat{\beta}_n^{\,{\mathrm{init}}}$ assumed to satisfy eq:conrateinitiallinf. Further, suppose that where ${C_{\mathrm{S,{\mathbf{X}}}}}>0$ is a positive constant, is also satisfied. Assume that the robustification parameter $\alpha_

Figures (3)

  • Figure 1: Left figure: Densities for normally distributed (blue) and conditionally normally distributed (red) errors. Right figure: Densities for skew t-distributed (blue) and conditionally skew t-distributed (red) errors.
  • Figure 2: Residuals of LASSO
  • Figure 3: Residuals of LASSO Huber estimate in the real-data example

Theorems & Definitions (41)

  • Remark 1: Order of approximation and Assumption \ref{['assfan2017']}
  • Theorem 1: Sign-consistency and rate in the $\ell_\infty$ norm under initial $\ell_\infty$ - bound
  • Theorem 2: Sign-consistency and rate in the $\ell_\infty$ norm under inital $\ell_2$ - bound
  • Remark 2
  • Lemma 1
  • Remark 3
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5: Solving the PDW construction
  • ...and 31 more