Data-driven configuration tuning of glmnet to balance accuracy and computation time

TL;DR

The paper tackles the problem that glmnet's default configuration can yield inaccurate lasso solutions on high-dimensional, correlated data due to suboptimal convergence and lambda grids. It proposes a data-driven framework that generates large-scale synthetic datasets, trains a neural predictor (glmnet-MLP) to map data characteristics and configuration to Solution Path Error ($\mathrm{SPE}$) and runtime, and then uses Pareto front optimization to select a glmnet configuration within a user-specified time budget $T_{\text{hope}}$. Key contributions include the construction of a large summary dataset (over 810k samples), a Bayesian-optimized neural architecture for predicting SPE and computation time, and a Pareto-front based tuning mechanism implemented in the glmnetconf R package, which can also choose between glmnet and LARS. Empirically, the method achieves accuracy close to the exact LARS solution while significantly reducing computation time, demonstrated in both synthetic simulations and compressed sensing tasks; future work extends training coverage and applies the framework to other GLMs. The approach provides a transparent, fast, and scalable way to balance accuracy and efficiency in regularized regression settings.

Abstract

glmnet is a widely adopted R package for lasso estimation due to its computational efficiency. Despite its popularity, glmnet sometimes yields solutions that are substantially different from the true ones because of the inappropriate default configuration of the algorithm. The accuracy of the obtained solutions can be improved by appropriately tuning the configuration. However, improving accuracy typically increases computational time, resulting in a trade-off between accuracy and computational efficiency. Therefore, it is essential to establish a systematic approach to determine appropriate configuration. To address this need, we propose a unified data-driven framework specifically designed to optimize the configuration by balancing the trade-off between accuracy and computational efficiency. We generate large-scale simulated datasets and apply glmnet under various configurations to obtain accuracy and computation time. Based on these results, we construct neural networks that predict accuracy and computation time from data characteristics and configuration. Given a new dataset, our framework uses the neural networks to explore the configuration space and derive a Pareto front that represents the trade-off between accuracy and computational cost. This front allows us to automatically identify the configuration that maximize accuracy under a user-specified time constraint. The proposed method is implemented in the R package 'glmnetconf', available at https://github.com/Shuhei-Muroya/glmnetconf.

Figures (9)

• Figure 1: The solution path for the same dataset by each package. The experimental setting is identical to that in Section \ref{['sec: numericalexp']}, with $N=1500, p=800, \rho=0.5$. For clarity, we display the solution path for only the first 10 coefficients to avoid visual congestion. The glmnet (default) denotes the estimator by glmnet using the default configuration, whereas glmnet (manual) denotes the estimator by glmnet whose configuration is manually optimized by the authors. LARS denotes the estimator by LARS algorithm, which provides the exact solution path and thus serves as a reference (ground truth). By comparison, the results of glmnet (manual) are seen to be close to that of LARS.
• Figure 2: Overview of the proposed framework. The process consists of two phases: Step 1 constructs a predictive model using a summary dataset generated from simulation parameters. Step 2 utilizes this trained model to predict performance metrics for a target dataset, selecting the best configuration that satisfies the time constraint $T_{\text{hope}}$.
• Figure 3: Visualization of the Pareto front derived from the glmnet-MLP for the same dataset used in Figure \ref{['fig:solutionpath']}. The horizontal and vertical axes represent the predicted SPE and computation time, respectively. The blue points represent the set of Pareto optimal solutions. From this set, the red triangle highlights the best configuration selected based on the user-specified time constraint ($T_{\text{hope}}=20~\mathrm{s}$), indicated by the horizontal dashed line. Under this constraint, the best configuration was identified as $(\tau^*, n_{\lambda}^*) = (1.159 \times 10^{-9}, 864)$.
• Figure 4: Comparison of prediction accuracy (RMSE) across different sample sizes $N$. The plot compares glmnet (default), glmnet (proposed) tuned with $T_\text{hope}=20~\mathrm{s}$ and LARS as the exact reference. The results are averaged over 100 simulation runs. The panels correspond to different combinations of the number of predictors $p$ and the correlation among the predictors $\rho$. Notably, the glmnet (proposed) consistently achieves accuracy comparable to the exact LARS solution across all settings.
• Figure 5: Comparison of computation time (seconds) across different sample sizes $N$. Similar to Figure \ref{['fig:simulation_rmse']}, this plot compares glmnet (default), glmnet (proposed) tuned with $T_\text{hope}=20~\mathrm{s}$, and LARS. The results are averaged over 100 simulation runs. The panels correspond to different combinations of the number of predictors $p$ and the correlation among the predictors $\rho$. The proposed method is not only significantly faster than LARS but also satisfies $T_\text{hope}$ in the majority of cases.

Theorems & Definitions (2)

• Definition 1: Weak dominance
• Definition 2: Pareto optimal solution and Pareto front