Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance

Lisha Chen, Quan Xiao, Ellen Hidemi Fukuda, Xinyi Chen, Kun Yuan, Tianyi Chen

TL;DR

This work tackles preference-guided multi-objective learning by formulating it as Optimization on the Pareto Set (OPS), a semivectorial bilevel problem. It introduces a smoothed merit function $v_{l,\tau}$ to scalarize vector-valued objectives and a penalty-based reformulation that connects solutions of the penalized problem to the original constrained OPS. A practical first-order algorithm, FOOPS, alternates between updating the Pareto-set variable $y$ and the decision variable $x$, with convergence guarantees under Hölderian error bounds and KL inequalities. Theoretical contributions include detailed properties of the merit function and its relations to weak Pareto optimality, while experiments across synthetic and real-world MOL tasks demonstrate competitive, preference-guided Pareto optimization. The framework offers a Hessian-free alternative to prior BLO methods with broad applicability to real systems requiring controlled trade-offs among objectives.

Abstract

Multi-objective learning under user-specified preference is common in real-world problems such as multi-lingual speech recognition under fairness. In this work, we frame such a problem as a semivectorial bilevel optimization problem, whose goal is to optimize a pre-defined preference function, subject to the constraint that the model parameters are weakly Pareto optimal. To solve this problem, we convert the multi-objective constraints to a single-objective constraint through a merit function with an easy-to-evaluate gradient, and then, we use a penalty-based reformulation of the bilevel optimization problem. We theoretically establish the properties of the merit function, and the relations of solutions for the penalty reformulation and the constrained formulation. Then we propose algorithms to solve the reformulated single-level problem, and establish its convergence guarantees. We test the method on various synthetic and real-world problems. The results demonstrate the effectiveness of the proposed method in finding preference-guided optimal solutions to the multi-objective problem.

Efficient First-Order Optimization on the Pareto Set for Multi-Objective Learning under Preference Guidance

TL;DR

This work tackles preference-guided multi-objective learning by formulating it as Optimization on the Pareto Set (OPS), a semivectorial bilevel problem. It introduces a smoothed merit function to scalarize vector-valued objectives and a penalty-based reformulation that connects solutions of the penalized problem to the original constrained OPS. A practical first-order algorithm, FOOPS, alternates between updating the Pareto-set variable and the decision variable , with convergence guarantees under Hölderian error bounds and KL inequalities. Theoretical contributions include detailed properties of the merit function and its relations to weak Pareto optimality, while experiments across synthetic and real-world MOL tasks demonstrate competitive, preference-guided Pareto optimization. The framework offers a Hessian-free alternative to prior BLO methods with broad applicability to real systems requiring controlled trade-offs among objectives.

Abstract

Multi-objective learning under user-specified preference is common in real-world problems such as multi-lingual speech recognition under fairness. In this work, we frame such a problem as a semivectorial bilevel optimization problem, whose goal is to optimize a pre-defined preference function, subject to the constraint that the model parameters are weakly Pareto optimal. To solve this problem, we convert the multi-objective constraints to a single-objective constraint through a merit function with an easy-to-evaluate gradient, and then, we use a penalty-based reformulation of the bilevel optimization problem. We theoretically establish the properties of the merit function, and the relations of solutions for the penalty reformulation and the constrained formulation. Then we propose algorithms to solve the reformulated single-level problem, and establish its convergence guarantees. We test the method on various synthetic and real-world problems. The results demonstrate the effectiveness of the proposed method in finding preference-guided optimal solutions to the multi-objective problem.

Paper Structure

This paper contains 41 sections, 41 theorems, 201 equations, 7 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem setup and preliminaries
  3. Problem reformulation
  4. A smoothed merit function and its properties
  5. Relation of different formulations
  6. Comparison with existing methods
  7. Algorithms and analysis
  8. Related works
  9. Experiments
  10. Conclusions
  11. Organization of the appendix.
  12. Extended discussion of related works
  13. Bilevel optimization
  14. Semivectorial BLO.
  15. Limitation of linear scalarization and preference as constraint
  16. ...and 26 more sections

Key Result

Proposition 3.3

Suppose Assumption assmp:weak_c_f holds. The merit function $v_{l,\tau}(x)$ defined in eq:vl satisfies the following properties: 1. $\bar{u}(x) - \tau \ln M \leq v_{0,\tau} (x) \leq \bar{u}(x)$. Furthermore, $\min_{x\in {\cal X}} v_{l,\tau}(x) = - \tau \ln M$. 2. If $x$ is weakly Pareto optimal, the

Figures (7)

  • Figure 1: Example on multi-lingual speech or language processing problem under user-specified preference guidance. The red curve represents the Pareto front, which is, informally, the set of objective values that achieve the best trade-offs among multiple objectives.
  • Figure 2: Illustration of $v_{l,\tau}$ and $\bar{u}$ with different values of $\tau$ and $l$ with $F(x) = ( \sqrt[6]{(x + \frac{1}{2})^2 + \frac{1}{8}}, \sqrt[6]{(x - \frac{1}{2})^2 + \frac{1}{8} } )^\top$.
  • Figure 3: Results of LS, FERERO, and FOOPS on Example \ref{['exmp:kkt_suboptimal']}. The black dashed lines is the preference defined by $H(x) =0$, or $f_0(x) = \|H(x)\|^2 = 0$. Green dots represent initial values, blue markers represent converged values for different methods.
  • Figure 4: Outputs (colored markers) and optimization trajectories (colored curves) of different methods when initial objectives are near the Pareto front. Dashed arrows with different colors represent different preferences.
  • Figure 5: Losses and accuracies of various methods with different preferences across three image datasets. The horizontal and vertical axes represent results for objective 1 and objective 2, respectively. Different colored dashed arrows indicate various preference vectors. Different markers denote the solutions obtained by different methods, with marker colors matching the preferences.
  • ...and 2 more figures

Theorems & Definitions (94)

  • Definition 2.1: Pareto dominance and optimality
  • Definition 2.2: Pareto stationarity e.g. ehrgott_multicriteria_2005
  • Definition 3.1: Weak convexity
  • Definition 3.2: Point strong quasar-convex functions hardt2018gd_linear_dy_quasar
  • Proposition 3.3: Properties of $v_{l,\tau}$
  • Definition 3.4: Subanalyticity Bierstone1988_Semianalytic
  • Definition 3.5: Global subanalyticity VanDenDriesMiller1996_ominimal
  • Definition 3.6: $(\varrho,\eta)$-Hölderian error bound
  • Lemma 3.7: Subanalyticity of ${\cal X}^*_{v_{l,\tau}}$ and $v_{l,\tau}(x)$
  • Definition 3.8: Global and local solutions
  • ...and 84 more