Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond the Lower Bound: Bridging Regret Minimization and Best Arm Identification in Lexicographic Bandits

Bo Xue, Yuanyu Wan, Zhichao Lu, Qingfu Zhang

TL;DR

This work introduces a unified framework for regret minimization and best arm identification under lexicographic (priority-based) multi-objective bandits. It develops two elimination-based algorithms, LexElim-Out and LexElim-In, that respect lexical priorities while leveraging cross-objective rewards to accelerate learning. Theoretical guarantees show LexElim-Out matches the best-known instance-dependent BAI bounds for the top objective, while LexElim-In achieves faster, cross-objective-aware rates, including minimax regret scaling $\widetilde{O}(\Lambda^i(\lambda)\sqrt{Kt})$ for each objective. Empirical results on synthetic data demonstrate superior performance over baselines, with LexElim-In particularly benefiting from information sharing across objectives. These results highlight the practical value of jointly optimizing RM and BAI in structured, multi-objective decision problems.

Abstract

In multi-objective decision-making with hierarchical preferences, lexicographic bandits provide a natural framework for optimizing multiple objectives in a prioritized order. In this setting, a learner repeatedly selects arms and observes reward vectors, aiming to maximize the reward for the highest-priority objective, then the next, and so on. While previous studies have primarily focused on regret minimization, this work bridges the gap between \textit{regret minimization} and \textit{best arm identification} under lexicographic preferences. We propose two elimination-based algorithms to address this joint objective. The first algorithm eliminates suboptimal arms sequentially, layer by layer, in accordance with the objective priorities, and achieves sample complexity and regret bounds comparable to those of the best single-objective algorithms. The second algorithm simultaneously leverages reward information from all objectives in each round, effectively exploiting cross-objective dependencies. Remarkably, it outperforms the known lower bound for the single-objective bandit problem, highlighting the benefit of cross-objective information sharing in the multi-objective setting. Empirical results further validate their superior performance over baselines.

Beyond the Lower Bound: Bridging Regret Minimization and Best Arm Identification in Lexicographic Bandits

TL;DR

This work introduces a unified framework for regret minimization and best arm identification under lexicographic (priority-based) multi-objective bandits. It develops two elimination-based algorithms, LexElim-Out and LexElim-In, that respect lexical priorities while leveraging cross-objective rewards to accelerate learning. Theoretical guarantees show LexElim-Out matches the best-known instance-dependent BAI bounds for the top objective, while LexElim-In achieves faster, cross-objective-aware rates, including minimax regret scaling for each objective. Empirical results on synthetic data demonstrate superior performance over baselines, with LexElim-In particularly benefiting from information sharing across objectives. These results highlight the practical value of jointly optimizing RM and BAI in structured, multi-objective decision problems.

Abstract

In multi-objective decision-making with hierarchical preferences, lexicographic bandits provide a natural framework for optimizing multiple objectives in a prioritized order. In this setting, a learner repeatedly selects arms and observes reward vectors, aiming to maximize the reward for the highest-priority objective, then the next, and so on. While previous studies have primarily focused on regret minimization, this work bridges the gap between \textit{regret minimization} and \textit{best arm identification} under lexicographic preferences. We propose two elimination-based algorithms to address this joint objective. The first algorithm eliminates suboptimal arms sequentially, layer by layer, in accordance with the objective priorities, and achieves sample complexity and regret bounds comparable to those of the best single-objective algorithms. The second algorithm simultaneously leverages reward information from all objectives in each round, effectively exploiting cross-objective dependencies. Remarkably, it outperforms the known lower bound for the single-objective bandit problem, highlighting the benefit of cross-objective information sharing in the multi-objective setting. Empirical results further validate their superior performance over baselines.

Paper Structure

This paper contains 19 sections, 15 theorems, 101 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Related Work
  4. RM.
  5. BAI.
  6. RM and BAI.
  7. MOB.
  8. Algorithms
  9. Warm-up: LexElim-Out
  10. Improved Algorithm: LexElim-In
  11. Cross-objective Acceleration.
  12. Experiments
  13. Conclusion and Future work
  14. Proof of Theorem \ref{['thm:1']}
  15. Proof of Theorem \ref{['thm:2']}
  16. ...and 4 more sections

Key Result

Theorem 1

Suppose that Eq. sub-Gaussian holds, define $\mathcal{S}(i) = \{a \in \mathcal{O}_*(i-1) \mid \Delta^i(a) > 0\}$ with $\mathcal{O}_*(0) = [K]$, and set $\gamma^i(\delta) = 64\log\left(\frac{392Km}{(\Delta^i(a))^2 \cdot \delta}\right)$. With probability at least $1-\delta$, for any objective $i\in[m]

Figures (3)

  • Figure 1: Cross-objective Acceleration
  • Figure 2: Regret Comparison of Our Algorithms versus PF-LEX and $\text{UCB}_\alpha$: $K=10$
  • Figure 3: Sample Complexity Comparison of Our Algorithms versus EGE and $\text{UCB}_\alpha$

Theorems & Definitions (17)

  • Definition 1: Lexicographic Order
  • Definition 2: Lex-optimal Arm
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Corollary 1
  • Theorem 4
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • ...and 7 more