Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asymptotics for estimating a diverging number of parameters -- with and without sparsity

Jana Gauss, Thomas Nagler

Abstract

We consider high-dimensional estimation problems where the number of parameters diverges with the sample size. General conditions are established for consistency, uniqueness, and asymptotic normality in both unpenalized and penalized estimation settings. The conditions are weak and accommodate a broad class of estimation problems, including ones with non-convex and group structured penalties. The wide applicability of the results is illustrated through diverse examples, including generalized linear models, multi-sample inference, and stepwise estimation procedures.

Asymptotics for estimating a diverging number of parameters -- with and without sparsity

Abstract

We consider high-dimensional estimation problems where the number of parameters diverges with the sample size. General conditions are established for consistency, uniqueness, and asymptotic normality in both unpenalized and penalized estimation settings. The conditions are weak and accommodate a broad class of estimation problems, including ones with non-convex and group structured penalties. The wide applicability of the results is illustrated through diverse examples, including generalized linear models, multi-sample inference, and stepwise estimation procedures.

Paper Structure

This paper contains 51 sections, 30 theorems, 230 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Unpenalized estimation
  3. Penalized estimation
  4. Outline
  5. Unpenalized Estimation
  6. Setup and Notation
  7. Consistency and Uniqueness
  8. Asymptotic Normality
  9. Penalized Estimation
  10. Penalization of Estimating Equations
  11. Examples and Conditions for the Penalty
  12. Estimation and selection consistency
  13. Notation and Assumptions
  14. Main results
  15. Mutual incoherence & irrepresentable condition
  16. ...and 36 more sections

Key Result

Theorem 1

Under assumption A:Cons1, the following holds with probability tending to 1:

Figures (1)

  • Figure 1: Two views on sparsity-inducing penalties: Left column: realization of $\Phi_n(\theta)$ (dashed line) and Lasso-penalized version (solid line). Right column: Penalized estimator as solution to $\Phi_n(\theta) \in \partial p_{\lambda}(\theta)$.

Theorems & Definitions (52)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Example 3.1: Lasso
  • Example 3.2: Elastic Net
  • Example 3.3: Group Lasso
  • Example 3.4: $\ell_q$ penalty
  • Example 3.5: SCAD
  • Example 3.6: MCP
  • Example 3.7: Fusion penalty
  • ...and 42 more