Data-Driven Preference Sampling for Pareto Front Learning

TL;DR

This work tackles learning the Pareto front in multi-objective neural networks by addressing the rigidity of fixed Dirichlet sampling for preference vectors. It introduces DDPS-MCMC, a data-driven framework that uses posterior information from training to adapt a mixture of Dirichlet distributions, enabling sampling of preference vectors that align with the actual Pareto front geometry, including disconnected fronts. Preference vectors are selectively retained via NSGA-II style sorting and updated with MCMC-based inference, producing adaptive parameters for the sampling distribution. Empirical results on synthetic benchmarks and multi-task image classification show DDPS-MCMC outperforms strong baselines in HV and IGD, demonstrating robust front coverage and flexibility across front shapes. The method offers a practical pathway to more accurate Pareto front estimation, with future work aimed at constrained MOO and more complex front structures.

Abstract

Pareto front learning is a technique that introduces preference vectors in a neural network to approximate the Pareto front. Previous Pareto front learning methods have demonstrated high performance in approximating simple Pareto fronts. These methods often sample preference vectors from a fixed Dirichlet distribution. However, no fixed sampling distribution can be adapted to diverse Pareto fronts. Efficiently sampling preference vectors and accurately estimating the Pareto front is a challenge. To address this challenge, we propose a data-driven preference vector sampling framework for Pareto front learning. We utilize the posterior information of the objective functions to adjust the parameters of the sampling distribution flexibly. In this manner, the proposed method can sample preference vectors from the location of the Pareto front with a high probability. Moreover, we design the distribution of the preference vector as a mixture of Dirichlet distributions to improve the performance of the model in disconnected Pareto fronts. Extensive experiments validate the superiority of the proposed method compared with state-of-the-art algorithms.

Figures (8)

• Figure 1: The blue region is the value range of the functions. We hope to find all the solutions of the Pareto front through different preference vectors.
• Figure 2: Data-driven Preference Sampling (DDPS) Flow Chart. In the ($t-1$)-th epoch of training, the preference vectors are sampled from the mixture of Dirichlet distributions $\mathcal{D}^{m}_{{t-1}}$ to form the set of loss values $\mathbb{D}_{t-1}$, which contains $N$ samples. After subset selection, the best $\lfloor \gamma pN \rfloor$ preference vectors are selected. The mixture of Dirichlet distributions is finally updated based on the posterior information.
• Figure 3: Sampling process of MCMC.
• Figure 4: Plot for three-objective problems. The first row is DTLZ7, the second row is DTLZ4, and the third row is DTLZ5. The yellow surfaces are the Pareto fronts, while the orange points are non-dominated solutions.
• Figure 5: Plot of two-objective problems. The first row is LZLZK, and the second row is ZDT3. The blue curves are the true Pareto front and the red points indicate the set of non-dominated solutions.