Look-Ahead Screening Rules for the Lasso
Johan Larsson
TL;DR
This paper addresses the computational burden of fitting the lasso across a regularization path in high-dimensional settings. It introduces look-ahead screening, a safe screening strategy based on Gap Safe rules, to discard predictors for a range of $\lambda$ values by exploiting the duality gap $G(\hat{\beta}_\lambda,\hat{\theta}_\lambda;\lambda)$. The approach yields faster solutions of the lasso path, with simulations demonstrating substantial time reductions, particularly at higher signal-to-noise ratios, and early path steps discarding the majority of predictors. The method is solver-agnostic and extensible to other safe and heuristic screening rules, offering practical gains for high-dimensional sparse regression tasks; public code is provided for reproducibility.
Abstract
The lasso is a popular method to induce shrinkage and sparsity in the solution vector (coefficients) of regression problems, particularly when there are many predictors relative to the number of observations. Solving the lasso in this high-dimensional setting can, however, be computationally demanding. Fortunately, this demand can be alleviated via the use of screening rules that discard predictors prior to fitting the model, leading to a reduced problem to be solved. In this paper, we present a new screening strategy: look-ahead screening. Our method uses safe screening rules to find a range of penalty values for which a given predictor cannot enter the model, thereby screening predictors along the remainder of the path. In experiments we show that these look-ahead screening rules outperform the active warm-start version of the Gap Safe rules.