None To Optima in Few Shots: Bayesian Optimization with MDP Priors

TL;DR

This work tackles the challenge of optimizing expensive black-box functions with very few evaluations. It introduces ProfBO, a Bayesian optimization framework that uses MDP priors to model optimization trajectories from related source tasks, embodied in a PFN surrogate and adapted via MAML for fast target-task performance, enabling effective search with $T \le 20$. The key contributions are the trajectory-prior modeling via MDPs, the integration of PFNs with meta-learning, and comprehensive demonstrations on real-world drug-discovery benchmarks and hyperparameter optimization showing superior few-shot performance and practical efficiency. The findings suggest that leveraging optimization-trajectory priors can substantially accelerate scientific discovery and engineering optimization in high-cost domains.

Abstract

Bayesian Optimization (BO) is an efficient tool for optimizing black-box functions, but its theoretical guarantees typically hold in the asymptotic regime. In many critical real-world applications such as drug discovery or materials design, where each evaluation can be very costly and time-consuming, BO becomes impractical for many evaluations. In this paper, we introduce the Procedure-inFormed BO (ProfBO) algorithm, which solves black-box optimization with remarkably few function evaluations. At the heart of our algorithmic design are Markov Decision Process (MDP) priors that model optimization trajectories from related source tasks, thereby capturing procedural knowledge on efficient optimization. We embed these MDP priors into a prior-fitted neural network and employ model-agnostic meta-learning for fast adaptation to new target tasks. Experiments on real-world Covid and Cancer benchmarks and hyperparameter tuning tasks demonstrate that ProfBO consistently outperforms state-of-the-art methods by achieving high-quality solutions with significantly fewer evaluations, making it ready for practical deployment.

Figures (9)

• Figure 1: Comparison of function predictions with only 3 observation points. $\textsc{ProfBO}$ models the true objective function curve significantly better than GP model and FSBO algorithm aea3f03299ff0cfea9b394f5559aa1c173f9876f.
• Figure 2: Overview of our $\textsc{ProfBO}$ framework. [Left] The BO loop: a PFN d88a5ae1673f2009704186acf2890163e6ddf4ca model pre-trained with a non-informative prior (e.g., GP) and fine-tuned with MDP priors from source tasks. Fine-tuning uses positional encoding and MAML finn2017modelagnosticmetalearningfastadaptation for better knowledge transfer. The PFN performs posterior inference on context $D$ via Transformer attention, with outputs interpreted as logits of a bar distribution. [Right] MDP prior training: for each meta-dataset $D^{(i)}$, a Deep Q-Network (DQN) mnih2013playingatarideepreinforcement policy generates optimization trajectories of the corresponding objectives. Right flowchart is attributed to Figure 1 in 774f5e2494f037302c58b7fa549c4f1cabf7295e.
• Figure 3: Strong performance of $\textsc{ProfBO}$ on Covid-B (five problems) and Cancer-B (two problems) compared with other baseline methods.
• Figure 4: [Left two] Strong performances of $\textsc{ProfBO}$ on HPO-B compared with other baseline methods. [Right two] Ablation study results of $\textsc{ProfBO}$ with MAML and positional encoding enabled ("MAML", "Pos") or not ("NoMAML", "NoPos"), showing both MAML and positional encoding are important technical components of $\textsc{ProfBO}$.
• Figure 5: Performance comparisons on Covid-B.