Asymptotically normal estimators in high-dimensional linear regression
Kou Fujimori, Koji Tsukuda
TL;DR
The paper develops a weak-convergence framework in the Hilbert space $\ell^2$ to obtain asymptotic normality for high-dimensional linear regression estimators, even when sparsity diverges and only a fixed set of moderate signals remains strong. By performing variable selection and then ordinary least squares on the selected model, the authors show that the $\ell^2$-valued estimator converges to a centered Gaussian field with a variance structure determined by the active moderate signals $T_\gamma$. They apply this result to linear-hypothesis testing, deriving a nonstandard null distribution for the test statistic as a finite weighted sum of independent $\chi^2$ variables and providing consistent plug-in estimators for the variance and eigenvalues to obtain asymptotically correct sizes. The framework enables standard asymptotic tools via the continuous mapping theorem and accommodates diverging dimensionality without requiring strong sparsity, with practical implications for inference in high-dimensional models.
Abstract
We establish asymptotic normality for estimators in high-dimensional linear regression by proving weak convergence in a separable Hilbert space, thereby enabling direct use of standard asymptotic tools, for example, the continuous mapping theorem. The approach allows the number of non-zero coefficients to grow, provided only a fixed number have moderate magnitude. As an application, we test linear hypotheses with a statistic whose null limit is a finite weighted sum of independent chi-squared variables, yielding plug-in critical values with asymptotically correct size.