Preferential Multi-Objective Bayesian Optimization

TL;DR

This work presents dueling scalarized Thompson sampling (DSTS), a multi-objective generalization of the popular dueling Thompson algorithm, a multi-objective generalization of the popular dueling Thompson algorithm that outperforms several benchmarks and proves that DSTS is asymptotically consistent.

Abstract

Preferential Bayesian optimization (PBO) is a framework for optimizing a decision-maker's latent preferences over available design choices. While preferences often involve multiple conflicting objectives, existing work in PBO assumes that preferences can be encoded by a single objective function. For example, in robotic assistive devices, technicians often attempt to maximize user comfort while simultaneously minimizing mechanical energy consumption for longer battery life. Similarly, in autonomous driving policy design, decision-makers wish to understand the trade-offs between multiple safety and performance attributes before committing to a policy. To address this gap, we propose the first framework for PBO with multiple objectives. Within this framework, we present dueling scalarized Thompson sampling (DSTS), a multi-objective generalization of the popular dueling Thompson algorithm, which may be of interest beyond the PBO setting. We evaluate DSTS across four synthetic test functions and two simulated exoskeleton personalization and driving policy design tasks, showing that it outperforms several benchmarks. Finally, we prove that DSTS is asymptotically consistent. As a direct consequence, this result provides, to our knowledge, the first convergence guarantee for dueling Thompson sampling in the PBO setting.

Figures (7)

• Figure 1: In this work, we extend preferential Bayesian optimization to the multi-objective setting through a novel self-sparring algorithm. Further, we apply this algorithm across four synthetic test problems and two practical applications (autonomous driving and exoskeleton locomotion). In contrast with existing approaches, our approach allows the decision-makers involved in the joint design task to efficiently explore the optimal trade-off between the conflicting objectives.
• Figure 2: Feasible region and Pareto front of the DTLZ2 test function.
• Figure 3: Our framework was demonstrated on six test problems: DTLZ1 (a), DTLZ2 (b), Vehicle Safety (c), Car Side Impact (d), Autonomous Driving (e), and Exoskeleton (f). Overall, our proposed method (DSTS) delivers the best performance. qMES and qParEGO exhibit a mixed performance, achieving good results in some test problems and poor results in others. The remaining methods, Random, PBO-DTS-IF, and qEHVI, consistently underperform DSTS.
• Figure 4: Simulation environments used in our test problems.
• Figure 5: Illustration of sampled designs for the DTLZ2 test function. These figures show that our proposed method (DSTS) provides a better exploration of the Pareto front than its competitors.

Theorems & Definitions (11)

• Theorem 1
• Lemma 1
• proof
• Lemma 2
• proof
• Theorem 1
• proof
• Theorem 2
• proof
• Theorem 3