Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Mosco sufficient condition for intrinsic stability of non-unique convex Empirical Risk Minimization

Karim Bounja, Lahcen Laayouni, Abdeljalil Sakat

TL;DR

This paper addresses intrinsic stability in non-unique convex ERM by adopting Painlevé–Kuratowski upper semicontinuity (PK-u.s.c.) of the solution correspondence as the central stability notion. It shows that Mosco perturbations together with locally bounded minimizers guarantee PK-u.s.c. and continuity of minimal values, with quadratic growth providing explicit deviation bounds for near-minimizers. It further demonstrates that strongly convex regularization enforces quadratic growth, delivering a practical stabilization mechanism and separating intrinsic ERM stability from solver-induced selection effects. The results offer a principled framework for understanding reproducibility and stability in high-dimensional, overparameterized settings and guide the design of robust ERM pipelines. Extending these guarantees to nonconvex ERM is identified as a natural future direction.

Abstract

Empirical risk minimization (ERM) stability is usually studied via single-valued outputs, while convex non-strict losses yield set-valued minimizers. We identify Painlevé-Kuratowski upper semicontinuity (PK-u.s.c.) as the intrinsic stability notion for the ERM solution correspondence (set-level Hadamard well-posedness) and a prerequisite to interpret stability of selections. We then characterize a minimal non-degenerate qualitative regime: Mosco-consistent perturbations and locally bounded minimizers imply PK-u.s.c., minimal-value continuity, and consistency of vanishing-gap near-minimizers. Quadratic growth yields explicit quantitative deviation bounds.

A Mosco sufficient condition for intrinsic stability of non-unique convex Empirical Risk Minimization

TL;DR

This paper addresses intrinsic stability in non-unique convex ERM by adopting Painlevé–Kuratowski upper semicontinuity (PK-u.s.c.) of the solution correspondence as the central stability notion. It shows that Mosco perturbations together with locally bounded minimizers guarantee PK-u.s.c. and continuity of minimal values, with quadratic growth providing explicit deviation bounds for near-minimizers. It further demonstrates that strongly convex regularization enforces quadratic growth, delivering a practical stabilization mechanism and separating intrinsic ERM stability from solver-induced selection effects. The results offer a principled framework for understanding reproducibility and stability in high-dimensional, overparameterized settings and guide the design of robust ERM pipelines. Extending these guarantees to nonconvex ERM is identified as a natural future direction.

Abstract

Empirical risk minimization (ERM) stability is usually studied via single-valued outputs, while convex non-strict losses yield set-valued minimizers. We identify Painlevé-Kuratowski upper semicontinuity (PK-u.s.c.) as the intrinsic stability notion for the ERM solution correspondence (set-level Hadamard well-posedness) and a prerequisite to interpret stability of selections. We then characterize a minimal non-degenerate qualitative regime: Mosco-consistent perturbations and locally bounded minimizers imply PK-u.s.c., minimal-value continuity, and consistency of vanishing-gap near-minimizers. Quadratic growth yields explicit quantitative deviation bounds.
Paper Structure (7 sections, 7 theorems, 18 equations)

This paper contains 7 sections, 7 theorems, 18 equations.

Table of Contents

  1. Introduction
  2. Variational setting
  3. Failure modes without boundedness or variational control
  4. Stability via Mosco continuity
  5. Quantitative stability from quadratic growth
  6. Strongly convex regularization as a stabilization mechanism
  7. Conclusion

Key Result

Proposition 1

Let $\mathcal{H}=\mathbb{R}$ and define, for $\varepsilon>0$, Then for every $\varepsilon>0$, $\mathcal{L}_{\varepsilon}$ is $C^{1}$ and convex, $\mathcal{C}_{\varepsilon} = \arg\min \mathcal{L}_{\varepsilon} = \left\{\frac{1}{\varepsilon}\right\}.$ Moreover, as $\varepsilon\downarrow 0$, $\mathcal{L}_\varepsilon$ converges pointwise to $\mathcal{L}_0(x)\equiv

Theorems & Definitions (23)

  • Definition 1: Admissible loss
  • Definition 2: Stability with respect to data perturbations
  • Definition 3: Outer limit and upper semicontinuity
  • Definition 4: Hadamard well-posedness at the set level
  • Remark 1
  • Proposition 1: Blow-up of minimizers under vanishing perturbations
  • proof
  • Lemma 1: Uniform boundedness excludes blow-up
  • proof
  • Proposition 2: Pointwise limits may destroy variational structure
  • ...and 13 more