Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning

Lisha Chen, AFM Saif, Yanning Shen, Tianyi Chen

TL;DR

This work introduces a Flexible framEwork for pREfeRence-guided multi-Objective learning (FERERO) by casting it as a constrained vector optimization problem, and introduces the first single-loop primal algorithm for constrained vector optimization to the authors' knowledge.

Abstract

Finding specific preference-guided Pareto solutions that represent different trade-offs among multiple objectives is critical yet challenging in multi-objective problems. Existing methods are restrictive in preference definitions and/or their theoretical guarantees. In this work, we introduce a Flexible framEwork for pREfeRence-guided multi-Objective learning (FERERO) by casting it as a constrained vector optimization problem. Specifically, two types of preferences are incorporated into this formulation -- the relative preference defined by the partial ordering induced by a polyhedral cone, and the absolute preference defined by constraints that are linear functions of the objectives. To solve this problem, convergent algorithms are developed with both single-loop and stochastic variants. Notably, this is the first single-loop primal algorithm for constrained vector optimization to our knowledge. The proposed algorithms adaptively adjust to both constraint and objective values, eliminating the need to solve different subproblems at different stages of constraint satisfaction. Experiments on multiple benchmarks demonstrate the proposed method is very competitive in finding preference-guided optimal solutions. Code is available at https://github.com/lisha-chen/FERERO/.

FERERO: A Flexible Framework for Preference-Guided Multi-Objective Learning

TL;DR

This work introduces a Flexible framEwork for pREfeRence-guided multi-Objective learning (FERERO) by casting it as a constrained vector optimization problem, and introduces the first single-loop primal algorithm for constrained vector optimization to the authors' knowledge.

Abstract

Finding specific preference-guided Pareto solutions that represent different trade-offs among multiple objectives is critical yet challenging in multi-objective problems. Existing methods are restrictive in preference definitions and/or their theoretical guarantees. In this work, we introduce a Flexible framEwork for pREfeRence-guided multi-Objective learning (FERERO) by casting it as a constrained vector optimization problem. Specifically, two types of preferences are incorporated into this formulation -- the relative preference defined by the partial ordering induced by a polyhedral cone, and the absolute preference defined by constraints that are linear functions of the objectives. To solve this problem, convergent algorithms are developed with both single-loop and stochastic variants. Notably, this is the first single-loop primal algorithm for constrained vector optimization to our knowledge. The proposed algorithms adaptively adjust to both constraint and objective values, eliminating the need to solve different subproblems at different stages of constraint satisfaction. Experiments on multiple benchmarks demonstrate the proposed method is very competitive in finding preference-guided optimal solutions. Code is available at https://github.com/lisha-chen/FERERO/.

Paper Structure

This paper contains 52 sections, 39 theorems, 198 equations, 11 figures, 12 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Problem Setup and A Meta Algorithm
  3. Problem setup and preliminaries
  4. Find the preference-guided direction
  5. A meta algorithm for preference-guided multi-objective learning
  6. Efficient Single-loop Algorithms
  7. Single-loop approximate algorithm
  8. Choice of relative preferences
  9. Related Works
  10. Experiments
  11. Synthetic data
  12. Real data
  13. Emotion recognition.
  14. Conclusions
  15. Notations
  16. ...and 37 more sections

Key Result

Lemma 1

For the subprogram eq:subp_original, the following holds: If $\theta$ is a local optimal solution with $A F(\theta) > 0$, then $d^*(\theta) = 0$, $\psi (\theta) = 0$. Otherwise, if $\theta$ is not a local optimal solution, then $d^*(\theta) \neq 0$, $\psi (\theta) < 0$, and when $\theta$ is feasible Let $\theta$ be a weak $C_A$-optimal solution, with $(A F(\theta))_m = 0$ for some $m\in [M]$. If t

Figures (11)

  • Figure 1: Illustration of preferences in different examples. The solid red curves represent the Pareto front, dashed lines represent preference constraints.
  • Figure 2: Illustration of $C_A$-dominance. The solid red curves are the Pareto fronts, green dots are the reference points, gray shaded regions are the set of objectives dominating the reference points, under different $C_A$ in (a) and (b).
  • Figure 3: Converging solutions (blue dots) and optimization trajectories (blue lines) on the objective space of different methods on synthetic objectives given in \ref{['eq:obj_synthetic1']}. Dashed arrows represent pre-specified preference vectors. The green dots represent initial objective values.
  • Figure 4: Outputs (colored markers) and optimization trajectories (colored lines) of different methods when initial objectives are near the Pareto front. Different colors represent different preferences.
  • Figure 5: Training losses and accuracies of various methods with different preferences across three image datasets. The horizontal and vertical axes represent results for objective 1 and objective 2, respectively. Different colored dashed arrows indicate various preference vectors. Different markers denote the solutions obtained by different methods, with marker colors matching the preferences.
  • ...and 6 more figures

Theorems & Definitions (86)

  • Definition 1: $C_A$-dominance ehrgott_multicriteria_2005jahn_vector_2011
  • Definition 2: $C_A$-optimal
  • Lemma 1
  • Lemma 2
  • Theorem 1: Convergence of the generic FERERO algorithm
  • Theorem 2: Convergence of the FERERO-SA algorithm
  • Definition 3: Proximal PL inequality
  • Theorem 3: Sharper convergence of the FERERO-SA algorithm
  • Definition 4: Cone
  • Lemma 3: Convex cone
  • ...and 76 more