Sparse classification with positive-confidence data in high dimensions

TL;DR

This work tackles high-dimensional binary classification under Positive-Confidence (Pconf) weak supervision, where only positive samples with confidence scores are available. It introduces a sparse-penalization framework that embeds Lasso, SCAD, and MCP penalties into the Pconf empirical risk, solved efficiently via a proximal gradient algorithm. Theoretical guarantees under Restricted Strong Convexity yield near-minimax sparse recovery rates, and the proximal-wrapper optimization achieves scalable performance with closed-form proximal maps. Empirical results on simulations and a sonar data application show predictive accuracy and variable-selection quality close to fully supervised methods, demonstrating the practical viability of sparse Pconf learning in $d\gg n$ regimes.

Abstract

High-dimensional learning problems, where the number of features exceeds the sample size, often require sparse regularization for effective prediction and variable selection. While established for fully supervised data, these techniques remain underexplored in weak-supervision settings such as Positive-Confidence (Pconf) classification. Pconf learning utilizes only positive samples equipped with confidence scores, thereby avoiding the need for negative data. However, existing Pconf methods are ill-suited for high-dimensional regimes. This paper proposes a novel sparse-penalization framework for high-dimensional Pconf classification. We introduce estimators using convex (Lasso) and non-convex (SCAD, MCP) penalties to address shrinkage bias and improve feature recovery. Theoretically, we establish estimation and prediction error bounds for the L1-regularized Pconf estimator, proving it achieves near minimax-optimal sparse recovery rates under Restricted Strong Convexity condition. To solve the resulting composite objective, we develop an efficient proximal gradient algorithm. Extensive simulations demonstrate that our proposed methods achieve predictive performance and variable selection accuracy comparable to fully supervised approaches, effectively bridging the gap between weak supervision and high-dimensional statistics.

Theorems & Definitions (6)

• Example 1
• Remark 1
• Theorem 1
• Proposition 1: High-probability bound on gradient $\|\nabla \widehat{R}_n(\beta^*)\|_\infty$
• proof : Proof of Theorem \ref{['thm_pconf_estimation_error']}
• proof : Proof of Proposition \ref{['lem:grad_concentration_bounded']}