Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic set-valued optimization and its application to robust learning

Tommaso Giovannelli, Jingfu Tan, Luis Nunes Vicente

Abstract

In this paper, we develop a stochastic set-valued optimization (SVO) framework tailored for robust machine learning. In the SVO setting, each decision variable is mapped to a set of objective values, and optimality is defined via set relations. We focus on SVO problems with hyperbox sets, which can be reformulated as multi-objective optimization (MOO) problems with finitely many objectives and serve as a foundation for representing or approximating more general mapped sets. Two special cases of hyperbox-valued optimization (HVO) are interval-valued (IVO) and rectangle-valued (RVO) optimization. We construct stochastic IVO/RVO formulations that incorporate subquantiles and superquantiles into the objective functions of the MOO reformulations, providing a new characterization for subquantiles. These formulations provide interpretable trade-offs by capturing both lower- and upper-tail behaviors of loss distributions, thereby going beyond standard empirical risk minimization and classical robust models. To solve the resulting multi-objective problems, we adopt stochastic multi-gradient algorithms and select a Pareto knee solution. In numerical experiments, the proposed algorithms with this selection strategy exhibit improved robustness and reduced variability across test replications under distributional shift compared with empirical risk minimization, while maintaining competitive accuracy.

Stochastic set-valued optimization and its application to robust learning

Abstract

In this paper, we develop a stochastic set-valued optimization (SVO) framework tailored for robust machine learning. In the SVO setting, each decision variable is mapped to a set of objective values, and optimality is defined via set relations. We focus on SVO problems with hyperbox sets, which can be reformulated as multi-objective optimization (MOO) problems with finitely many objectives and serve as a foundation for representing or approximating more general mapped sets. Two special cases of hyperbox-valued optimization (HVO) are interval-valued (IVO) and rectangle-valued (RVO) optimization. We construct stochastic IVO/RVO formulations that incorporate subquantiles and superquantiles into the objective functions of the MOO reformulations, providing a new characterization for subquantiles. These formulations provide interpretable trade-offs by capturing both lower- and upper-tail behaviors of loss distributions, thereby going beyond standard empirical risk minimization and classical robust models. To solve the resulting multi-objective problems, we adopt stochastic multi-gradient algorithms and select a Pareto knee solution. In numerical experiments, the proposed algorithms with this selection strategy exhibit improved robustness and reduced variability across test replications under distributional shift compared with empirical risk minimization, while maintaining competitive accuracy.
Paper Structure (33 sections, 7 theorems, 67 equations, 7 figures, 1 algorithm)

This paper contains 33 sections, 7 theorems, 67 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Robust learning motivation.
  3. A brief SVO literature review.
  4. Our contributions.
  5. Background
  6. Notation and basic definitions
  7. Partial order relations between vectors
  8. Partial order relation between sets
  9. Vector-valued optimization and stochastic multi-objective optimization
  10. Set-valued mappings and set-valued optimization
  11. Vectorization for set-valued optimization
  12. Hyperbox-valued optimization
  13. Stochastic hyperbox-valued optimization
  14. Superquantiles and subquantiles
  15. Stochastic interval-valued optimization
  16. ...and 18 more sections

Key Result

Theorem 2.1

Let $S(x)+K$ and $S(x)-K$ be closed and convex for all $x \in X$. Then, $\bar{x} \in X$ is a strictly minimal solution to problem prob:svo if and only if $\bar{x}$ is a strictly efficient solution to the following VVO problem

Figures (7)

  • Figure 1: Partial order relations in $\mathbb{R}^2$ for $K = \mathbb{R}^2_+$.
  • Figure 2: Key concepts in SVO.
  • Figure 3: $S(x)$ for problem \ref{['prob:stochastic_ivo']}.
  • Figure 4: Vectorization theorem for IVO.
  • Figure 5: Adult dataset results for IVO.
  • ...and 2 more figures

Theorems & Definitions (8)

  • Theorem 2.1: Vectorization theorem
  • Proposition 3.1: Comparison of hyperboxes
  • Theorem 3.1: Vectorization for HVO
  • Corollary 3.1
  • Corollary 3.2
  • Remark 5.1
  • Lemma A.1
  • Theorem A.2