Offline Model-Based Optimization: Comprehensive Review

TL;DR

The paper surveys offline model-based optimization (MBO), formalizing single- and multi-objective problem settings, and reframing offline optimization as a conditional generation task. It dissects two main research threads—surrogate modeling for robust extrapolation and generative modeling for efficient exploration of high-dimensional design spaces—highlighting where they complement each other and how they interact. Benchmarks are categorized into synthetic, real-world, scientific design, and ML-model tasks, with evaluation metrics emphasizing usefulness, diversity, and novelty, and with an emphasis on epistemic uncertainty and potential reward hacking. The discussion identifies core challenges—benchmark realism, uncertainty quantification, graphical surrogates, advanced generative methods, and AI safety implications—and outlines directions such as robust benchmarking, graphical modeling, and LLM alignment applications to steer future work.

Abstract

Offline optimization is a fundamental challenge in science and engineering, where the goal is to optimize black-box functions using only offline datasets. This setting is particularly relevant when querying the objective function is prohibitively expensive or infeasible, with applications spanning protein engineering, material discovery, neural architecture search, and beyond. The main difficulty lies in accurately estimating the objective landscape beyond the available data, where extrapolations are fraught with significant epistemic uncertainty. This uncertainty can lead to objective hacking(reward hacking), exploiting model inaccuracies in unseen regions, or other spurious optimizations that yield misleadingly high performance estimates outside the training distribution. Recent advances in model-based optimization(MBO) have harnessed the generalization capabilities of deep neural networks to develop offline-specific surrogate and generative models. Trained with carefully designed strategies, these models are more robust against out-of-distribution issues, facilitating the discovery of improved designs. Despite its growing impact in accelerating scientific discovery, the field lacks a comprehensive review. To bridge this gap, we present the first thorough review of offline MBO. We begin by formalizing the problem for both single-objective and multi-objective settings and by reviewing recent benchmarks and evaluation metrics. We then categorize existing approaches into two key areas: surrogate modeling, which emphasizes accurate function approximation in out-of-distribution regions, and generative modeling, which explores high-dimensional design spaces to identify high-performing designs. Finally, we examine the key challenges and propose promising directions for advancement in this rapidly evolving field including safe control of superintelligent systems.

Figures (1)

• Figure 1: Paper overview.