Multi-Fidelity Bayesian Optimization for Nash Equilibria with Black-Box Utilities

TL;DR

This work tackles learning approximate pure Nash equilibria in a centralized Stackelberg-like setting where players' utilities are expensive and only accessible through black-box evaluations. It introduces MF-UCB-PNE, a multi-fidelity Bayesian optimization policy that alternates between low-cost exploratory sampling and high-fidelity evaluation to identify incentive-compatible configurations under a query budget. The approach relies on a multi-output Gaussian process surrogate with an auto-regressive fidelity structure and an information-gain-per-cost acquisition to bound regret and achieve asymptotic no-regret performance. Theoretical regret bounds are complemented by experiments on synthetic games and real-world-like wireless and MAC tasks, showing MF-UCB-PNE consistently finds higher-quality approximate $\epsilon^*$-PNE within budget. This framework enables cost-effective coordination in open, software-defined systems with heterogeneous, conflicting objectives.

Abstract

Modern open and softwarized systems -- such as O-RAN telecom networks and cloud computing platforms -- host independently developed applications with distinct, and potentially conflicting, objectives. Coordinating the behavior of such applications to ensure stable system operation poses significant challenges, especially when each application's utility is accessible only via costly, black-box evaluations. In this paper, we consider a centralized optimization framework in which a system controller suggests joint configurations to multiple strategic players, representing different applications, with the goal of aligning their incentives toward a stable outcome. To model this interaction, we formulate a Stackelberg game in which the central optimizer lacks access to analytical utility functions and instead must learn them through sequential, multi-fidelity evaluations. To address this challenge, we propose MF-UCB-PNE, a novel multi-fidelity Bayesian optimization strategy that leverages a budget-constrained sampling process to approximate pure Nash equilibrium (PNE) solutions. MF-UCB-PNE systematically balances exploration across low-cost approximations with high-fidelity exploitation steps, enabling efficient convergence to incentive-compatible configurations. We provide theoretical and empirical insights into the trade-offs between query cost and equilibrium accuracy, demonstrating the effectiveness of MF-UCB-PNE in identifying effective equilibrium solutions under limited cost budgets.

Figures (9)

• Figure 1: An example of settings captured by the framework studied in this paper: In an open system such as an O-RAN architecture, applications, known as xApps, are deployed simultaneously at a controller. The xApps have different utility functions, and thus their different preferences may cause a conflict. For example, an xApp targeting energy efficiency may attempt to shut down inefficient radio units, while other xApps may attempt to maximize throughput, causing a conflict on the use of the available transmission resources. The controller can seek joint configurations for the two apps that provide limited incentives for the two apps to deviate from it.
• Figure 2: This paper studies a setting in which a central optimizer aims at identifying an approximate pure Nash equilibrium (PNE) for a multi-player strategic game, while having access only to multi-fidelity estimators of expensive-to-evaluate black-box utility functions for the $N$ players. At any time $t$, the central optimizer selects an action profile $\mathbf{x}_{t}$ and a fidelity vector $\mathbf{m}_{t}=[m_{1,t},...,m_{N,t}]$. As a result, the optimizer receives noisy utility feedback $y_{n,t}^{(m_{n,t})}$ about the corresponding utility value $u^{(m_{n,t})}_n(\mathbf{x}_{t})$ at fidelity level $m_{n,t}$ incurring a query cost $\lambda^{(m_{n,t})}$, for all players $n\in\mathcal{N}$. The goal is to approach a solution in the $\epsilon^*$-PNE set \ref{['eq: epsilon pne']}, where $\epsilon^*$ is the smallest achievable dissatisfaction level, while abiding by a total query budget $\Lambda$.
• Figure 3: Block diagram illustrating the operation of the proposed MF-UCB-PNE policy MF-UCB-PNE operates via a sequence of episodes $\mathcal{E}_j$ consisting of an exploration phase and of an evaluation phase.
• Figure 4: Visualization of MF-UCB-PNE optimization trajectories against a heat map of each player's utility function. The total query budget is set to $\Lambda=32$, and the threshold in \ref{['eq: exploring condition 2']} is set to $\eta=0.5$. The indexes $(t,m_{n,t})$ associated to each point in the heat map describe the time step $t$ increasing the counter across episodes and the fidelity decision $m_{n,t}$ assigned to player $n$ at time step $t$. The $\epsilon^*$-PNE is marked as a red star. The initial action at time $t=1$ and the actions made in evaluation phases are shown as red squares, while decisions made in the exploration phases are marked as black circles. Dashed black arrows follow the updates across time steps, while solid red arrows connect the actions obtained at the evaluation phases at the end of each episode.
• Figure 5: Visualization of PE (left) and UCB-PNE (right) optimization trajectories with total query budget $\Lambda=32$. The indices $(t,m_{n,t})$ associated to each point describe the time step $t$ and the maximum-fidelity $m_{n,t}=M=2$ assigned to all players at time step $t$. The $\epsilon^*$-PNE is marked as a red star.

Theorems & Definitions (8)

• Definition 1: Episode
• Definition 2: Episode Regret
• Lemma 1: Uniform Error Bound in RKHS
• Lemma 2: Dissatisfaction Coverage
• Theorem 1: Regret Bound of MF-UCB-PNE
• proof
• Corollary 1: No-Regret of MF-UCB-PNE
• proof