Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Scalarizations for Sublinear Hypervolume Regret

Qiuyi Zhang

TL;DR

It is shown that hypervolume scalarizations with uniformly random weights achieves an optimal sublinear hypervolume regret bound of $O(T^{-1/k})$, with matching lower bounds that preclude any algorithm from doing better asymptotically.

Abstract

Scalarization is a general, parallizable technique that can be deployed in any multiobjective setting to reduce multiple objectives into one, yet some have dismissed this versatile approach because linear scalarizations cannot explore concave regions of the Pareto frontier. To that end, we aim to find simple non-linear scalarizations that provably explore a diverse set of $k$ objectives on the Pareto frontier, as measured by the dominated hypervolume. We show that hypervolume scalarizations with uniformly random weights achieves an optimal sublinear hypervolume regret bound of $O(T^{-1/k})$, with matching lower bounds that preclude any algorithm from doing better asymptotically. For the setting of multiobjective stochastic linear bandits, we utilize properties of hypervolume scalarizations to derive a novel non-Euclidean analysis to get regret bounds of $\tilde{O}( d T^{-1/2} + T^{-1/k})$, removing unnecessary $\text{poly}(k)$ dependencies. We support our theory with strong empirical performance of using non-linear scalarizations that outperforms both their linear counterparts and other standard multiobjective algorithms in a variety of natural settings.

Optimal Scalarizations for Sublinear Hypervolume Regret

TL;DR

It is shown that hypervolume scalarizations with uniformly random weights achieves an optimal sublinear hypervolume regret bound of , with matching lower bounds that preclude any algorithm from doing better asymptotically.

Abstract

Scalarization is a general, parallizable technique that can be deployed in any multiobjective setting to reduce multiple objectives into one, yet some have dismissed this versatile approach because linear scalarizations cannot explore concave regions of the Pareto frontier. To that end, we aim to find simple non-linear scalarizations that provably explore a diverse set of objectives on the Pareto frontier, as measured by the dominated hypervolume. We show that hypervolume scalarizations with uniformly random weights achieves an optimal sublinear hypervolume regret bound of , with matching lower bounds that preclude any algorithm from doing better asymptotically. For the setting of multiobjective stochastic linear bandits, we utilize properties of hypervolume scalarizations to derive a novel non-Euclidean analysis to get regret bounds of , removing unnecessary dependencies. We support our theory with strong empirical performance of using non-linear scalarizations that outperforms both their linear counterparts and other standard multiobjective algorithms in a variety of natural settings.
Paper Structure (18 sections, 16 theorems, 36 equations, 11 figures)

This paper contains 18 sections, 16 theorems, 36 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Problem Setting and Notation
  4. Scalarizations for Multiobjective Optimization
  5. Hypervolume Scalarizations
  6. Lower Bounds and Optimality
  7. Multiobjective Stochastic Linear Bandits
  8. Experiments
  9. Synthetic Optimization
  10. Stochastic Linear Bandits
  11. BBOB Functions
  12. Linear Bandit Setup and Algorithm
  13. Missing Proofs
  14. Proofs of Lipschitz Properties
  15. Proofs for Linear Bandits
  16. ...and 3 more sections

Key Result

Theorem 1

Let $\mathbf{Y}_T = \{y_1,..., y_T\}$ be a set of $T$ points in $\mathbb{R}^k$ such that $y_i \in \arg \underset{y\in\mathcal{Y}}{\max} s^\textsc{HV}_{\lambda_i}(y)$ with $\lambda_i \sim \mathcal{S}_+$ randomly drawn i.i.d. from an uniform distribution and $s^{\textsc{HV}}$ are hypervolume scalariza where $\mathcal{Y}^\star$ is the Pareto frontier and $\mathcal{HV}$ is the hypervolume function. Fu

Figures (11)

  • Figure 1: Left: Comparisons of the scalarized minimization solutions with various weights with convex and non-convex Pareto fronts. The colors represent different weights; the dots are scalarized optima and the dotted lines represent level curves. Linear scalarization does not have an optima in the concave region of the Pareto front for any set of weights, but the non-linear scalarization, with its sharper level curves, can discover the whole Pareto front (Figure from emmerich2018tutorial). Right: The dotted red lines represent the level curves of the hypervolume scalarization with $\lambda = v$, discovering $b$, whereas the linear scalarization would prefer $a$ or $c$. Furthermore, the optima is exactly the Pareto point that is in the direction of $v$.
  • Figure 2: The hypervolume scalarization taken with respect to a direction $\lambda=w$ corresponds to a differential area element within dominated hypervolume and averaging over random directions is analogous to integrating over the dominated hypervolume in polar coordinates. We exploit this fact to show that by choosing the maximizers of $T$ random hypervolume scalarizers, we quickly converge to the hypervolume of the Pareto frontier at an optimal rate of $O(T^{-1/k})$. Figure from song2024vizier
  • Figure 3: Comparisons of multiple scalarizations for the synthetic concave Pareto frontier given by $z = \exp(-x-y)$. The hypervolume regret for Linear is constant, and the Hypervolume enjoys a faster regret convergence rate than the Chebyshev.
  • Figure 4: Comparisons of the cumulative hypervolume plots with some anti-correlated $\theta$. When the output dimension increase, there is a clearer advantage to using the hypervolume scalarization over the linear and Chebyshev scalarization. We find that the boxed weight distribution does consistently worse than the uniform distribution.
  • Figure 5: Comparisons of the hypervolume indicator and the optimization fronts with BBOB functions. The left plot tracks the dominated hypervolume as a function of trials that were evaluated. The blue/orange dots represent the frontier points of the UCB-HV/EHVI algorithms respectively, over 5 repeats. Especially in high dimensions, EHVI tends overly concentrate on points in the middle of the frontier, limiting its hypervolume gain, while hypervolume scalarizations produce more diverse points.
  • ...and 6 more figures

Theorems & Definitions (17)

  • Theorem 1: Informal Restatement of thm:hypervolume_asymptotic and thm:lowerhv
  • Theorem 2: Informal Restatement of thm:hypervolumeregret
  • Definition 3
  • Proposition 4: emmerich2018tutorial
  • Lemma 5
  • Lemma 6: Hypervolume in Expectation golovin2020random
  • Theorem 7: Sublinear Hypervolume Regret
  • Theorem 8: Hypervolume Regret Lower Bound
  • Corollary 9
  • Lemma 10
  • ...and 7 more