High Dimensional Gaussian and Bootstrap Approximations in Generalized Linear Models
Mayukh Choudhury, Debraj Das
TL;DR
This paper analyzes GLM estimators in high dimensions, establishing a uniform Gaussian approximation over convex sets and Euclidean balls in Regime I with precise growth rates for $d$ relative to $n$ and extending to bootstrap when the covariance is unknown. It further shows that when $d$ grows much faster than $n$, Lasso with Variable Selection Consistency cannot preserve a Gaussian limit for the active coefficients, motivating a Perturbation Bootstrap approach (PB-Lasso) that yields a Berry–Esseen-type bound for uniform convex-set convergence. The two-regime framework yields concrete rates and validation via simulations and real-data applications, demonstrating that PB-based inference remains reliable under both moderate and ultra-high dimensional GLMs. The work integrates Bahadur representations, Gaussian isoperimetric inequalities, and high-dimensional CLT techniques to provide rigorous, practically relevant inferential tools for GLMs in modern, large-$d$ contexts. Overall, the proposed PB methodology offers robust uncertainty quantification for GLM estimators and Lasso-based models in high-dimensional regimes, with demonstrated applicability to real-world biomedical and engineering datasets.
Abstract
Generalized Linear Models (GLMs) extend ordinary linear regression by linking the mean of the response variable to covariates through appropriate link functions. This paper investigates the asymptotic behavior of GLM estimators when the parameter dimension $d$ grows with the sample size $n$. In the first part, we establish Gaussian approximation results for the distribution of a properly centered and scaled GLM estimator uniformly over class of convex sets and Euclidean balls. Using high-dimensional results from Fang and Koike (2024) for the leading Bahadur term, bounding remainder terms as in He and Shao (2000), and applying Nazarov's (2003) Gaussian isoperimetric inequality, we show that Gaussian approximation holds when $d = o(n^{2/5})$ for convex sets and $d = o(n^{1/2})$ for Euclidean balls-the best possible rates matching those for high-dimensional sample means. We further extend these results to the bootstrap approximation when the covariance matrix is unknown. In the second part, when $d>>n$, a natural question is to answer whether all covariates are equally important. To answer that, we employ sparsity in GLM through the Lasso estimator. While Lasso is widely used for variable selection, it cannot achieve both Variable Selection Consistency (VSC) and $n^{1/2}$-consistency simultaneously (Lahiri, 2021). Under the regime ensuring VSC, we show that Gaussian approximation for the Lasso estimator fails. To overcome this, we propose a Perturbation Bootstrap (PB) approach and establish a Berry-Esseen type bound for its approximation uniformly over class of convex sets. Simulation studies confirm the strong finite-sample performance of the proposed method.