Preference-Guided Diffusion for Multi-Objective Offline Optimization

TL;DR

This work tackles offline multi-objective optimization by proposing PGD-MOO, a diffusion-based generator guided by a preference classifier that predicts Pareto dominance between design pairs. The method avoids per-objective surrogates, instead leveraging dominance-based guidance and diversity-aware training to extrapolate toward the Pareto front while maintaining solution spread. Empirical results across synthetic, engineering, and NAS benchmarks show PGD-MOO is competitive with forward surrogate-based methods and often superior to other inverse approaches, with strong diversity metrics. The approach offers a scalable, data-efficient pathway for discovering diverse Pareto-optimal designs in settings where online evaluations are infeasible.

Abstract

Offline multi-objective optimization aims to identify Pareto-optimal solutions given a dataset of designs and their objective values. In this work, we propose a preference-guided diffusion model that generates Pareto-optimal designs by leveraging a classifier-based guidance mechanism. Our guidance classifier is a preference model trained to predict the probability that one design dominates another, directing the diffusion model toward optimal regions of the design space. Crucially, this preference model generalizes beyond the training distribution, enabling the discovery of Pareto-optimal solutions outside the observed dataset. We introduce a novel diversity-aware preference guidance, augmenting Pareto dominance preference with diversity criteria. This ensures that generated solutions are optimal and well-distributed across the objective space, a capability absent in prior generative methods for offline multi-objective optimization. We evaluate our approach on various continuous offline multi-objective optimization tasks and find that it consistently outperforms other inverse/generative approaches while remaining competitive with forward/ surrogate-based optimization methods. Our results highlight the effectiveness of classifier-guided diffusion models in generating diverse and high-quality solutions that approximate the Pareto front well.

Figures (5)

• Figure 1: A schematic representation of the proposed preference guided diffusion approach for offline multi-objective optimization.
• Figure 2: Generalization of the preference model on regions unseen in the training data on the ZDT2 task zitzler2000comparison. The preference model gives good prediction of Pareto dominance between the reference design (in black) with other designs (in pink and green). Pink indicates that the preference model predicts these designs to be dominated by the reference design and green indicates that these designs are predicted as dominating the reference design. The figure on the right is a zoomed-in version of the left, excluding the training data (in blue).
• Figure 3: Plot of the samples from our preference-guided diffusion model (in green) on the ZDT2 task zitzler2000comparison at different timesteps of denoising. Convergence of samples close to the Pareto front (in red) outside of the training data (blue) highlights the importance of preference guidance.
• Figure 4: Ablation study of (a) different guidance weights $w$ on both hypervolume and $\Delta$-spread metrics (b) various diversity criteria (Crowding, SubCrowding, No Diversity and Hypervolume) on the $\Delta$-spread metric. All evaluation done with ZDT subtask on 256 sampled designs across 5 random seeds.
• Figure 5: Plot of the samples from diffusion model (in green) on different 2-objective tasks. Top row shows results for synthetic set of benchmarks, while the bottom row shows results for RE engineering suite. Blue dots correspond to training data and red dot corresponds to the true Pareto front. The blue dots are omitted for the bottom row for clarity. Results for ZDT2 is available in \ref{['fig:denoising-sp']}.

Theorems & Definitions (2)

• Definition 2.1: Pareto Dominance
• Definition 2.2: Pareto Optimality and Pareto Front