Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust estimation with Lasso when outputs are adversarially contaminated

Takeyuki Sasai, Hironori Fujisawa

TL;DR

The proof is improved and the proof is given a sharper convergence rate than Nguyen and Tran (2012) , when the number of outliers is larger, to use some specific properties of the Huber function.

Abstract

We consider robust estimation when outputs are adversarially contaminated. Nguyen and Tran (2012) proposed an extended Lasso for robust parameter estimation and then they showed the convergence rate of the estimation error. Recently, Dalalyan and Thompson (2019) gave some useful inequalities and then they showed a faster convergence rate than Nguyen and Tran (2012). They focused on the fact that the minimization problem of the extended Lasso can become that of the penalized Huber loss function with $L_1$ penalty. The distinguishing point is that the Huber loss function includes an extra tuning parameter, which is different from the conventional method. We give the proof, which is different from Dalalyan and Thompson (2019) and then we give the same convergence rate as Dalalyan and Thompson (2019). The significance of our proof is to use some specific properties of the Huber function. Such techniques have not been used in the past proofs.

Robust estimation with Lasso when outputs are adversarially contaminated

TL;DR

The proof is improved and the proof is given a sharper convergence rate than Nguyen and Tran (2012) , when the number of outliers is larger, to use some specific properties of the Huber function.

Abstract

We consider robust estimation when outputs are adversarially contaminated. Nguyen and Tran (2012) proposed an extended Lasso for robust parameter estimation and then they showed the convergence rate of the estimation error. Recently, Dalalyan and Thompson (2019) gave some useful inequalities and then they showed a faster convergence rate than Nguyen and Tran (2012). They focused on the fact that the minimization problem of the extended Lasso can become that of the penalized Huber loss function with penalty. The distinguishing point is that the Huber loss function includes an extra tuning parameter, which is different from the conventional method. We give the proof, which is different from Dalalyan and Thompson (2019) and then we give the same convergence rate as Dalalyan and Thompson (2019). The significance of our proof is to use some specific properties of the Huber function. Such techniques have not been used in the past proofs.

Paper Structure

This paper contains 26 sections, 57 theorems, 203 equations.

Table of Contents

  1. Introduction
  2. Main theorem and outline of proof
  3. Main theorem
  4. Outline of the proof of the main theorem
  5. Preliminary
  6. Relation between $\|\beta^* -\hat{\beta}\|_1$ and $\|\Sigma^{\frac{1}{2}}(\beta^*-\hat{\beta})\|_2$
  7. Restricted eigenvalue condition
  8. Relation between $\|\beta^* -\hat{\beta}\|_1$ and $\|\Sigma^{\frac{1}{2}}(\beta^*-\hat{\beta})\|_2$
  9. Inequalities related to the estimation error
  10. Proof of Proposition \ref{['p:coe-re']}
  11. Evaluation of $C_{cut}$
  12. Outline of the proofs of the key propositions
  13. Proof of main theorem
  14. Notation
  15. Theorem
  16. ...and 11 more sections

Key Result

Theorem 2.1

Suppose that $\Sigma$ satisfies the restricted eigenvalue condition $\mathrm{RE}(s,5,\kappa)$ (cf. Definition def:RE). Assume that $\delta$ is sufficiently small and $n$ is sufficiently large. Let where $C_{\lambda_o}$ is an appropriate numerical constant, Assume that $\lambda_o$ and $\lambda_s$ satisfy where $C_{n,\delta}$ is given later. Under some additional conditions, with probability at l

Theorems & Definitions (100)

  • Theorem 2.1
  • Remark 2.1
  • Corollary 2.1
  • Theorem 2.2: NguTra2012Robust
  • Definition 3.1: Gaussian width
  • Lemma 3.1: Theorem 2.5 of BouLugMas2013concentration
  • Lemma 3.2
  • proof
  • Lemma 3.3
  • proof
  • ...and 90 more