Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimization

Qiang Zhang, Ruida Zhou, Yang Shen, Tie Liu

TL;DR

This work reframes offline optimization as sampling from a score-based generative model and directly optimizes a target distribution that concentrates on high-objective regions by learning a weight function. Leveraging a PAC lower bound, it jointly learns a weight function and a score-based DDPM, enabling principled tradeoffs between utility and learnability. The approach yields a distribution-dependent surrogate for the optimization objective and demonstrates robust, competitive improvements on standard offline benchmarks, including Design-Bench tasks. The contributions provide a theoretically grounded alternative to surrogate-model methods and offer practical benefits for generating high-quality designs in data-constrained settings.

Abstract

This paper considers the problem of offline optimization, where the objective function is unknown except for a collection of ``offline" data examples. While recent years have seen a flurry of work on applying various machine learning techniques to the offline optimization problem, the majority of these work focused on learning a surrogate of the unknown objective function and then applying existing optimization algorithms. While the idea of modeling the unknown objective function is intuitive and appealing, from the learning point of view it also makes it very difficult to tune the objective of the learner according to the objective of optimization. Instead of learning and then optimizing the unknown objective function, in this paper we take on a less intuitive but more direct view that optimization can be thought of as a process of sampling from a generative model. To learn an effective generative model from the offline data examples, we consider the standard technique of ``re-weighting", and our main technical contribution is a probably approximately correct (PAC) lower bound on the natural optimization objective, which allows us to jointly learn a weight function and a score-based generative model. The robustly competitive performance of the proposed approach is demonstrated via empirical studies using the standard offline optimization benchmarks.

From Function to Distribution Modeling: A PAC-Generative Approach to Offline Optimization

TL;DR

This work reframes offline optimization as sampling from a score-based generative model and directly optimizes a target distribution that concentrates on high-objective regions by learning a weight function. Leveraging a PAC lower bound, it jointly learns a weight function and a score-based DDPM, enabling principled tradeoffs between utility and learnability. The approach yields a distribution-dependent surrogate for the optimization objective and demonstrates robust, competitive improvements on standard offline benchmarks, including Design-Bench tasks. The contributions provide a theoretically grounded alternative to surrogate-model methods and offer practical benefits for generating high-quality designs in data-constrained settings.

Abstract

This paper considers the problem of offline optimization, where the objective function is unknown except for a collection of ``offline" data examples. While recent years have seen a flurry of work on applying various machine learning techniques to the offline optimization problem, the majority of these work focused on learning a surrogate of the unknown objective function and then applying existing optimization algorithms. While the idea of modeling the unknown objective function is intuitive and appealing, from the learning point of view it also makes it very difficult to tune the objective of the learner according to the objective of optimization. Instead of learning and then optimizing the unknown objective function, in this paper we take on a less intuitive but more direct view that optimization can be thought of as a process of sampling from a generative model. To learn an effective generative model from the offline data examples, we consider the standard technique of ``re-weighting", and our main technical contribution is a probably approximately correct (PAC) lower bound on the natural optimization objective, which allows us to jointly learn a weight function and a score-based generative model. The robustly competitive performance of the proposed approach is demonstrated via empirical studies using the standard offline optimization benchmarks.
Paper Structure (20 sections, 3 theorems, 50 equations, 7 figures, 4 tables, 2 algorithms)

This paper contains 20 sections, 3 theorems, 50 equations, 7 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Wasserstein distance
  4. Denoising diffusion probabilistic model
  5. Generalization bound for weighted learning
  6. Main Results
  7. Algorithm
  8. Experimental Results
  9. A toy example
  10. Benchmark datasets
  11. Concluding Remarks
  12. Proof of Technical Results
  13. Proof of Lemma \ref{['lemma']}
  14. Proof of Theorem \ref{['theorem']}
  15. Detailed Experimental Results
  16. ...and 5 more sections

Key Result

Lemma 1

For any given $\tilde{w}$, with probability $\geq 1-\delta$ we have for any$\theta \in \Theta$ where is the empirical weighted loss over the training dataset ${\bm{x}}_{[m]}$, is the empirical variance of $\tilde{w}$ over ${\bm{x}}_{[m]}$, and is the empirical Rademacher complexity with respect to the parameter family $\Theta$ over ${\bm{x}}_{[m]}$.

Figures (7)

  • Figure 1: A toy example: The objective function and the initial samples.
  • Figure 2: A toy example: The optimized samples and the learned weight function for different values of $\alpha$. Top: Optimized samples; Bottom: Learned weight function.
  • Figure 3: Optimized samples (with trainable weight function) for different choices of the hyper-parameter $\alpha$.
  • Figure 4: Learned Weight function $w_{\phi^*}$ for different choices of the hyper-parameter $\alpha$.
  • Figure 5: Optimized samples with predefined weight function for different choices of the hyper-parameter $\psi$.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Lemma 1
  • Proposition 1
  • proof
  • Theorem 1