Uniform Asymptotics and Confidence Regions Based on the Adaptive Lasso with Partially Consistent Tuning

TL;DR

This paper studies the adaptive Lasso with componentwise tuning in a low-dimensional linear model, allowing some penalties to be zero and introducing a moving-parameter framework. It derives both consistency properties and an asymptotic distribution that accounts for tuning heterogeneity, showing that the confidence regions based on the estimator are governed by a deterministic benchmark set $\mathcal{M}$ whose open supersets achieve asymptotic coverage $1$ while small boundary relaxations yield coverage $0$. The analysis reveals that the shape and dimension of the confidence region depend on the regressor Gram matrix $C$, the tuning deviations $\lambda^0$ and $\psi$, and the potential randomness in the limit, providing a generalization of earlier one-dimensional results to general non-orthogonal designs. The results inform valid inference procedures for adaptive Lasso-based confidence regions and illustrate the impact of partial tuning on both the geometry and the coverage properties of these regions.

Abstract

We consider the adaptive Lasso estimator with componentwise tuning in the framework of a low-dimensional linear regression model. In our setting, at least one of the components is penalized at the rate of consistent model selection and certain components may not be penalized at all. We perform a detailed study of the consistency properties and the asymptotic distribution which includes the effects of componentwise tuning within a so-called moving-parameter framework. These results enable us to explicitly provide a set $\mathcal{M}$ such that every open superset acts as a confidence set with uniform asymptotic coverage equal to 1, whereas removing an arbitrarily small open set along the boundary yields a confidence set with uniform asymptotic coverage equal to 0. The shape of the set $\mathcal{M}$ depends on the regressor matrix as well as the deviations within the componentwise tuning parameters. Our findings can be viewed as a broad generalization of Pötscher & Schneider (2009, 2010) who considered distributional properties and confidence intervals based on components of the adaptive Lasso estimator for the case of orthogonal regressors.

Figures (3)

• Figure 1: An example for the set $\mathcal{M}$ with uniform tuning in $p=2$ dimensions.
• Figure 2: An example for the set $\mathcal{M}$ with uniform tuning and $p=3$ dimensions. The three-dimensional set is depicted in (\ref{['subfig:conf_3d']}) whereas its two-dimensional projections are shown in (\ref{['subfig:conf_proj']}).
• Figure 3: An example of the set $\mathcal{M}$ with partial tuning and $p=3$ dimensions. The first component is not penalized resulting in the set being part of a two-dimensional subspace.

Theorems & Definitions (30)

• Lemma 1: Relationship to LS estimator
• Corollary 2: Equivalence to LS estimator
• Proposition 3: Pointwise convergence rate
• Proposition 4: Uniform convergence rate
• Theorem 5: Consistency in parameter estimation
• Theorem 6: Consistency in model selection
• Remark
• Theorem 7: Asymptotic distribution
• Proposition 8: Set of minimizers
• Theorem 9: Confidence regions