R2 v2: The Pareto-compliant R2 Indicator for Better Benchmarking in Bi-objective Optimization

TL;DR

This paper advances unary benchmarking metrics for bi-objective optimization by deriving and implementing an exact, Pareto-compliant R2 indicator under a continuous, uniform distribution of Tchebycheff utilities. It provides an $O(N \log N)$ algorithm to compute R2 for a set of $N$ nondominated solutions and an incremental update scheme for dynamic benchmarking scenarios, along with the concept of exclusive contribution to preserve Pareto compliance. The authors characterize exact R2 values for ideal/nadir points, linear fronts, and simple convex/concave shapes, and contrast them with the discretized R2 commonly used in practice, demonstrating improved consistency and avoidance of plateau effects seen with hypervolume benchmarking. The work positions the exact R2 indicator as a practical, Pareto-compliant alternative to HV when an utopian reference point is natural, enabling efficient benchmarking and potential integration into optimization heuristics.

Abstract

In multi-objective optimization, set-based quality indicators are a cornerstone of benchmarking and performance assessment. They capture the quality of a set of trade-off solutions by reducing it to a scalar number. One of the most commonly used set-based metrics is the R2 indicator, which describes the expected utility of a solution set to a decision-maker under a distribution of utility functions. Typically, this indicator is applied by discretizing the latter distribution, yielding a weakly Pareto-compliant indicator. In consequence, adding a nondominated or dominating solution to a solution set may -- but does not have to -- improve the indicator's value. In this paper, we reinvestigate the R2 indicator under the premise that we have a continuous, uniform distribution of (Tchebycheff) utility functions. We analyze its properties in detail, demonstrating that this continuous variant is indeed Pareto-compliant -- that is, any beneficial solution will improve the metric's value. Additionally, we provide efficient computational procedures that (a) compute this metric for bi-objective problems in $\mathcal O (N \log N)$, and (b) can perform incremental updates to the indicator whenever solutions are added to (or removed from) the current set of solutions, without needing to recompute the indicator for the entire set. As a result, this work contributes to the state-of-the-art Pareto-compliant unary performance metrics, such as the hypervolume indicator, offering an efficient and promising alternative.

Figures (9)

• Figure 1: Left: Illustration of the HV indicator. Right: If no point dominating the HV is found, the minimal distance to the region of interest (dotted) can be used as an additional indicator. The combined indicator is, however, not Pareto-compliant anymore.
• Figure 2: Illustration of level sets of the Tchebycheff utility for five different weight vectors $w$. The utility value $u_w(y)$ is determined by the surface that the level set touches first: At vertical surfaces (see left and center image in the top row), the $u_w(y)$ is determined by the $f_1$ value while at horizontal surfaces (see bottom row) $u_w(y)$ depends on the $f_2$ value of $y$. At $y$ (the top right figure) both $w_1 y_1$ and $w_2 y_2$ are identical. Note: Weight vectors are illustrated to point towards their equilibrium between both objectives, i.e., $w_1 f_1 = w_2 f_2$.
• Figure 3: An axis-parallel (vertical) line segment between $y$ and $y'$ with utility $\color{Orange} u(y_1, [y_2, y_2'])$. For horizontal segments, the computation is analogous.
• Figure 4: Special case of a Pareto front consisting of a single point $y$. The corresponding R2 utility thus is $R2(\{y\}) = \color{Orange}u(y_1, [y_2, \infty))\color{black} + \color{NavyBlue}u(y_2, [y_1, \infty))$.
• Figure 5: Integration ranges surrounding a solution $y^{(n)}$. $y^{(n-1)}$, $y^{(n)}$, and $y^{(n+1)}$ are consecutive points in the solution set. The utility of the vertical (orange) segment is determined by the $\color{Orange} f_1$ value of $y^{(n)}$, while the utility of points on the horizontal (blue) segment is dependent on its $\color{NavyBlue} f_2$ value. The length of the segments depends on the neighbors of $y^{(n)}$ in the solution set.