Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimistic Games for Combinatorial Bayesian Optimization with Application to Protein Design

Melis Ilayda Bal, Pier Giuseppe Sessa, Mojmir Mutny, Andreas Krause

TL;DR

This work tackles the challenge of optimizing expensive black-box functions over large combinatorial, unstructured spaces such as protein design. It introduces GameOpt, a cooperative-game BO framework that computes equilibria of the Upper Confidence Bound acquisition to propose informative evaluation points, enabling scalable optimization without exhaustive search. The authors provide a sample-complexity bound for reaching approximate equilibria and validate the method on multiple real-world protein design datasets, where GameOpt consistently finds higher-fitness variants faster and with more diverse exploration than baselines. The approach leverages GP surrogates with per-variable embeddings, demonstrates robustness to limited initial data, and offers practical advantages over latent-space or traditional discrete optimization methods. Overall, GameOpt enables efficient, scalable exploration of massive combinatorial spaces with direct applicability to protein engineering and related domains.

Abstract

Bayesian optimization (BO) is a powerful framework to optimize black-box expensive-to-evaluate functions via sequential interactions. In several important problems (e.g. drug discovery, circuit design, neural architecture search, etc.), though, such functions are defined over large $\textit{combinatorial and unstructured}$ spaces. This makes existing BO algorithms not feasible due to the intractable maximization of the acquisition function over these domains. To address this issue, we propose $\textbf{GameOpt}$, a novel game-theoretical approach to combinatorial BO. $\textbf{GameOpt}$ establishes a cooperative game between the different optimization variables, and selects points that are game $\textit{equilibria}$ of an upper confidence bound acquisition function. These are stable configurations from which no variable has an incentive to deviate$-$ analog to local optima in continuous domains. Crucially, this allows us to efficiently break down the complexity of the combinatorial domain into individual decision sets, making $\textbf{GameOpt}$ scalable to large combinatorial spaces. We demonstrate the application of $\textbf{GameOpt}$ to the challenging $\textit{protein design}$ problem and validate its performance on four real-world protein datasets. Each protein can take up to $20^{X}$ possible configurations, where $X$ is the length of a protein, making standard BO methods infeasible. Instead, our approach iteratively selects informative protein configurations and very quickly discovers highly active protein variants compared to other baselines.

Optimistic Games for Combinatorial Bayesian Optimization with Application to Protein Design

TL;DR

This work tackles the challenge of optimizing expensive black-box functions over large combinatorial, unstructured spaces such as protein design. It introduces GameOpt, a cooperative-game BO framework that computes equilibria of the Upper Confidence Bound acquisition to propose informative evaluation points, enabling scalable optimization without exhaustive search. The authors provide a sample-complexity bound for reaching approximate equilibria and validate the method on multiple real-world protein design datasets, where GameOpt consistently finds higher-fitness variants faster and with more diverse exploration than baselines. The approach leverages GP surrogates with per-variable embeddings, demonstrates robustness to limited initial data, and offers practical advantages over latent-space or traditional discrete optimization methods. Overall, GameOpt enables efficient, scalable exploration of massive combinatorial spaces with direct applicability to protein engineering and related domains.

Abstract

Bayesian optimization (BO) is a powerful framework to optimize black-box expensive-to-evaluate functions via sequential interactions. In several important problems (e.g. drug discovery, circuit design, neural architecture search, etc.), though, such functions are defined over large spaces. This makes existing BO algorithms not feasible due to the intractable maximization of the acquisition function over these domains. To address this issue, we propose , a novel game-theoretical approach to combinatorial BO. establishes a cooperative game between the different optimization variables, and selects points that are game of an upper confidence bound acquisition function. These are stable configurations from which no variable has an incentive to deviate analog to local optima in continuous domains. Crucially, this allows us to efficiently break down the complexity of the combinatorial domain into individual decision sets, making scalable to large combinatorial spaces. We demonstrate the application of to the challenging problem and validate its performance on four real-world protein datasets. Each protein can take up to possible configurations, where is the length of a protein, making standard BO methods infeasible. Instead, our approach iteratively selects informative protein configurations and very quickly discovers highly active protein variants compared to other baselines.
Paper Structure (52 sections, 1 theorem, 7 equations, 18 figures, 6 tables, 8 algorithms)

This paper contains 52 sections, 1 theorem, 7 equations, 18 figures, 6 tables, 8 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Problem Statement and Background
  4. Problem statement
  5. Gaussian Processes (GPs)
  6. Bayesian Optimization (BO)
  7. GameOpt algorithm
  8. A form of local optimality
  9. Price of Anarchy
  10. Equilibrium finding subroutines
  11. Iterative best responses
  12. Multiplicative weights updates
  13. Related work
  14. Sample-Complexity Guarantees
  15. Application to Protein Design
  16. ...and 37 more sections

Key Result

Theorem 4.2

Assume $f$ satisfies the regularity assumptions of Section sec:problem-statement-background, and GameOpt is run with confidence width $\beta_t = 2n\log\left(\sup_{i\in \mathcal{N}}|\mathcal{X}_i|\frac{t^2\pi^2}{6\delta}\right)$. Then, with probability at least $1-\delta$ and for a given accuracy $\e

Figures (18)

  • Figure 1: Illustration of GameOpt. GameOpt defines a game among the decision variables, where game rewards are represented by the upper confidence bound (UCB) function. This decouples the combinatorial decision space into individual decision sets and allows GameOpt to tractably compute game equilibria. These can be thought of as local optima of the AF in unstructured domains.
  • Figure 2: Convergence speed of methods in terms of log fitness value of the best-so-far protein across BO iterations, under batch size $B =5$. Results are averaged over replications initiated with different training sets: $100$ protein variants for Halogenase and GB1(4), and $1000$ protein variants for other domains. Error bars are interquartile ranges averaged over replications. In all experiments GameOpt with Ibr and Hedge subroutines discover better protein sequences at a much faster rate.
  • Figure 3: Sampled batch diversity relative to past, measured via mean Hamming distance between executed and proposed variants from the previous iteration (pairwise distance), under batch size $B=5$. Results are averaged over replications, and error bars show interquartile ranges. In all experiments GameOpt consistently samples a rather diverse batch of evaluation points w.r.t. past proposed variants. This enhanced exploration of the search space contributes to its strong performance compared to the baseline methods.
  • Figure 4: Performance results for the sampled batch diversity w.r.t past measured via mean Hamming distance between the executed variant and the closest initial point from the training set, under batch size $B=5$. Each point on each line is the average of multiple replications initiated with different training sets having $100$ variants for GB1(4) & Halogenase and $1000$ for GB1(55) & GFP. Similarly, error bars are interquartile ranges averaged over replications. In all experiments, GameOpt explores significantly faster than the baseline methods.
  • Figure 5: The effect of training set size on the performance under setting GB1(4), $n=4$, $18$ reps. GP-UCB mitigates saturating behavior when leveraging a more informative initial GP surrogate model. In contrast, GameOpt showcases resilience in overcoming the limitations associated with a GP model trained with a limited amount of data.
  • ...and 13 more figures

Theorems & Definitions (3)

  • Definition 3.1: Nash equilibrium Nash1951
  • Definition 4.1: $\epsilon$-approximate Nash equilibrium
  • Theorem 4.2: Sample complexity of GameOpt