Pathwise optimization for bridge-type estimators and its applications

TL;DR

This work tackles pathwise optimization for bridge-type estimators with multiple adaptive $\ell^q$ penalties ($q_i\in(0,1]$), addressing nonconvexity and nondifferentiability through two nonconvex optimization schemes: accelerated proximal gradient and blockwise proximal alternating minimization. It establishes convergence to critical points, analyzes path consistency, and demonstrates oracle-like properties under suitable conditions. The framework is applied to penalized GLMs and high-frequency stochastic differential equations, with simulations showing potential gains in variable selection and estimation accuracy relative to LASSO and one-step LLA. These contributions offer a computationally viable route to sparse, nonconvex penalized estimation in both classical regression and time-series/diffusion contexts, with practical implications for model selection and interpretation in complex stochastic systems.

Abstract

Sparse parametric models are of great interest in statistical learning and are often analyzed by means of regularized estimators. Pathwise methods allow to efficiently compute the full solution path for penalized estimators, for any possible value of the penalization parameter $λ$. In this paper we deal with the pathwise optimization for bridge-type problems; i.e. we are interested in the minimization of a loss function, such as negative log-likelihood or residual sum of squares, plus the sum of $\ell^q$ norms with $q\in(0,1]$ involving adpative coefficients. For some loss functions this regularization achieves asymptotically the oracle properties (such as the selection consistency). Nevertheless, since the objective function involves nonconvex and nondifferentiable terms, the minimization problem is computationally challenging. The aim of this paper is to apply some general algorithms, arising from nonconvex optimization theory, to compute efficiently the path solutions for the adaptive bridge estimator with multiple penalties. In particular, we take into account two different approaches: accelerated proximal gradient descent and blockwise alternating optimization. The convergence and the path consistency of these algorithms are discussed. In order to assess our methods, we apply these algorithms to the penalized estimation of diffusion processes observed at discrete times. This latter represents a recent research topic in the field of statistics for time-dependent data.

Figures (6)

• Figure 1:
• Figure 2: Comparison between LASSO and Bridge path.
• Figure 3: Comparison of test errors as a function of normalized penalization parameter $\lambda / \lambda_{\max}$ for bridge (solid line), LLA (dashed), LASSO (dotted). Figure B is a zoom of Figure A around the minimum of the curves.
• Figure 4: A simulated sample path of the solution to \ref{['eq:sim-sde-vec']}
• Figure 5: Model selection proportions for $n=10^3$ as a function of the normalized penalization parameter.

Theorems & Definitions (14)

• Definition 1
• Remark 4.1
• Remark 4.2
• Remark 5.1
• Proposition 1
• proof
• Remark 5.2
• Lemma 1
• proof
• Theorem 1: Path consistency