Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Group Lasso Regularized Rank Regression with Data-Driven Parameter Determination

Meixia Lin, Meijiao Shi, Yunhai Xiao, Qian Zhang

TL;DR

Robust, high-dimensional rank regression is achieved by replacing the squared loss with a Wilcoxon rank loss, $L(X\beta - y)$, and enforcing group sparsity via a group lasso penalty, $\lambda\Psi(\beta)$. The authors introduce a data-driven rule for selecting $\lambda$, prove a finite-sample error bound for the estimator, and develop a proximal augmented Lagrangian method (PALM) with a semismooth Newton solver to efficiently solve the resulting optimization problem. The approach yields strong robustness to heavy-tailed noise and scalable performance on large datasets, outperforming cross-validated group lasso baselines in both accuracy and computation time. These contributions advance robust high-dimensional regression with structured sparsity and provide a practical, theoretically grounded toolkit for practitioners.

Abstract

High-dimensional regression often suffers from heavy-tailed noise and outliers, which can severely undermine the reliability of least-squares based methods. To improve robustness, we adopt a non-smooth Wilcoxon score based rank objective and incorporate structured group sparsity regularization, a natural generalization of the lasso, yielding a group lasso regularized rank regression method. By extending the tuning-free parameter selection scheme originally developed for the lasso, we introduce a data-driven, simulation-based tuning rule and further establish a finite-sample error bound for the resulting estimator. On the computational side, we develop a proximal augmented Lagrangian method for solving the associated optimization problem, which eliminates the singularity issues encountered in existing methods, thereby enabling efficient semismooth Newton updates for the subproblems. Extensive numerical experiments demonstrate the robustness and effectiveness of our proposed estimator against alternatives, and showcase the scalability of the algorithm across both simulated and real-data settings.

Efficient Group Lasso Regularized Rank Regression with Data-Driven Parameter Determination

TL;DR

Robust, high-dimensional rank regression is achieved by replacing the squared loss with a Wilcoxon rank loss, , and enforcing group sparsity via a group lasso penalty, . The authors introduce a data-driven rule for selecting , prove a finite-sample error bound for the estimator, and develop a proximal augmented Lagrangian method (PALM) with a semismooth Newton solver to efficiently solve the resulting optimization problem. The approach yields strong robustness to heavy-tailed noise and scalable performance on large datasets, outperforming cross-validated group lasso baselines in both accuracy and computation time. These contributions advance robust high-dimensional regression with structured sparsity and provide a practical, theoretically grounded toolkit for practitioners.

Abstract

High-dimensional regression often suffers from heavy-tailed noise and outliers, which can severely undermine the reliability of least-squares based methods. To improve robustness, we adopt a non-smooth Wilcoxon score based rank objective and incorporate structured group sparsity regularization, a natural generalization of the lasso, yielding a group lasso regularized rank regression method. By extending the tuning-free parameter selection scheme originally developed for the lasso, we introduce a data-driven, simulation-based tuning rule and further establish a finite-sample error bound for the resulting estimator. On the computational side, we develop a proximal augmented Lagrangian method for solving the associated optimization problem, which eliminates the singularity issues encountered in existing methods, thereby enabling efficient semismooth Newton updates for the subproblems. Extensive numerical experiments demonstrate the robustness and effectiveness of our proposed estimator against alternatives, and showcase the scalability of the algorithm across both simulated and real-data settings.

Paper Structure

This paper contains 16 sections, 10 theorems, 85 equations, 2 figures, 8 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Data-Driven Regularization with Theoretical Guarantees
  3. Simulation-Based Parameter
  4. Finite-Sample Bound on Estimation Error
  5. Proof of Theorem \ref{['th:consistency']}
  6. An Efficient Superlinearly Convergent Algorithm
  7. The Proximal Augmented Lagrangian Method and Its Convergence
  8. A Semismooth Newton Method for the Subproblem the Subproblem
  9. Implementation Details of the Semismooth Newton Method
  10. Numerical Experiments
  11. Overview of Experimental Design
  12. Statistical Performance of Group Lasso Regularized Rank Regression
  13. Scalability of PALM for Group Lasso Regularized Rank Regression
  14. Benchmarking PALM on L1-Regularized Rank Regression
  15. Conclusion
  16. ...and 1 more sections

Key Result

Lemma 1

Under the linear model mod:linear regression, the subgradient of $L_0(\cdot)$ in eq: def_L0 at $\gamma = 0$ is where $\xi = 2r - (n+1)$ and $r$ is uniformly distributed over all permutations of $[n]$.

Figures (2)

  • Figure 1: PALM computation time vs. dimension $p$ for fixed $n=500$ (left) and $n=2000$ (right).
  • Figure 2: PALM computation time vs. sample size $n$ at fixed dimensions $p=8000$ (left) and $p=100000$ (right).

Theorems & Definitions (18)

  • Lemma 1
  • Remark 1
  • Remark 2
  • Theorem 1
  • Corollary 1
  • Lemma 2
  • Lemma 3
  • proof : of Theorem \ref{['th:consistency']}
  • Remark 3
  • Theorem 2
  • ...and 8 more