Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Estimation of sparse linear regression coefficients under $L$-subexponential covariates

Takeyuki Sasai

TL;DR

This paper presents an error bound identical to that obtained for Gaussian random vectors, up to constant factors, without requiring stronger conditions, even when the covariates are drawn from an $L$-subexponential random vector.

Abstract

We tackle estimating sparse coefficients in a linear regression when the covariates are sampled from an $L$-subexponential random vector. This vector belongs to a class of distributions that exhibit heavier tails than Gaussian random vector. Previous studies have established error bounds similar to those derived for Gaussian random vectors. However, these methods require stronger conditions than those used for Gaussian random vectors to derive the error bounds. In this study, we present an error bound identical to the one obtained for Gaussian random vectors up to constant factors without imposing stronger conditions, when the covariates are drawn from an $L$-subexponential random vector. Interestingly, we employ an $\ell_1$-penalized Huber regression, which is known for its robustness against heavy-tailed random noises rather than covariates. We believe that this study uncovers a new aspect of the $\ell_1$-penalized Huber regression method.

Estimation of sparse linear regression coefficients under $L$-subexponential covariates

TL;DR

This paper presents an error bound identical to that obtained for Gaussian random vectors, up to constant factors, without requiring stronger conditions, even when the covariates are drawn from an -subexponential random vector.

Abstract

We tackle estimating sparse coefficients in a linear regression when the covariates are sampled from an -subexponential random vector. This vector belongs to a class of distributions that exhibit heavier tails than Gaussian random vector. Previous studies have established error bounds similar to those derived for Gaussian random vectors. However, these methods require stronger conditions than those used for Gaussian random vectors to derive the error bounds. In this study, we present an error bound identical to the one obtained for Gaussian random vectors up to constant factors without imposing stronger conditions, when the covariates are drawn from an -subexponential random vector. Interestingly, we employ an -penalized Huber regression, which is known for its robustness against heavy-tailed random noises rather than covariates. We believe that this study uncovers a new aspect of the -penalized Huber regression method.
Paper Structure (18 sections, 5 theorems, 55 equations)

This paper contains 18 sections, 5 theorems, 55 equations.

Table of Contents

  1. Introduction
  2. Related work, our estimation method and key idea
  3. Some definitions
  4. Related work
  5. Other related work
  6. Our estimation method and key idea
  7. Our main result and key propositions
  8. Proofs of the main propositions
  9. Preparation
  10. Proof of Proposition \ref{['p:main:upper']}
  11. Proof of Proposition \ref{['p:main:lower']}
  12. Proof of the main theorem
  13. Proof of Theorem \ref{['t:main']}: Step1
  14. Proof of Theorem \ref{['t:main']}: Step 1(a)
  15. Proof of Theorem \ref{['t:main']}: Step 1(b)
  16. ...and 3 more sections

Key Result

Theorem 3.1

Suppose that Assumption a:intro holds. Let $c_1$ and $c_2$ be numerical constants, which are defined in Propositions p:main:upper and p:main:lower. For the tuning parameters $\lambda_o$ and $\lambda_s$, we assume that where $c_s$ denotes a numerical constant such that $c_s \geq 5c_1$. Assume that $r_1 =3\sqrt{s}r_2$ and $r_2= 5L\lambda_o \sqrt{n}(c_1+c_2+c_s) (r_{d,s}+r_\delta)$, and $n$ is suf

Theorems & Definitions (11)

  • Definition 2.1: $\psi_\alpha$-norm
  • Definition 2.2: $L$-subGaussian and $L$-subexponential random vector
  • Definition 2.3: $L$-subGaussian and $\sigma$-subexponential random variable
  • Theorem 3.1
  • Proposition 3.1
  • Proposition 3.2
  • Lemma A.1
  • proof
  • Definition A.1: $\gamma_\alpha$-functional Dir2015Tail
  • Lemma A.2
  • ...and 1 more