Pareto Set Prediction Assisted Bilevel Multi-objective Optimization

TL;DR

This work tackles the computational bottleneck of solving bilevel multi-objective optimization problems (BLMOPs) by predicting the lower-level Pareto set (PS) for a given upper-level solution. A simple data transformation plus a helper variable $r \in [0,1]$ enables training a feedforward neural network to map UL inputs $\mathbf{x}_u$ (and $r$) to LL PS, which is then integrated into a bilevel evolutionary framework (PSP-BLEMO) to seed LL search or bypass it entirely. The method includes a variable association analysis (VAA) to handle LL variables that affect UL objectives, and maintains diversity via DSS within an FF/ND/CD environment selection, with an IGD-based termination criterion. Empirical results on ten benchmark BLMOPs show PSP-BLEMO achieves competitive or superior ILD (IGD) performance while using far fewer LL evaluations, particularly on deceptive instances; a one-shot variant further reduces training overhead with modest performance loss in some settings. The approach offers a practical route to efficient BLMOP optimization and opens avenues for higher-objective extensions and more sophisticated data selection strategies.

Abstract

Bilevel optimization problems comprise an upper level optimization task that contains a lower level optimization task as a constraint. While there is a significant and growing literature devoted to solving bilevel problems with single objective at both levels using evolutionary computation, there is relatively scarce work done to address problems with multiple objectives (BLMOP) at both levels. For black-box BLMOPs, the existing evolutionary techniques typically utilize nested search, which in its native form consumes large number of function evaluations. In this work, we propose to reduce this expense by predicting the lower level Pareto set for a candidate upper level solution directly, instead of conducting an optimization from scratch. Such a prediction is significantly challenging for BLMOPs as it involves one-to-many mapping scenario. We resolve this bottleneck by supplementing the dataset using a helper variable and construct a neural network, which can then be trained to map the variables in a meaningful manner. Then, we embed this initialization within a bilevel optimization framework, termed Pareto set prediction assisted evolutionary bilevel multi-objective optimization (PSP-BLEMO). Systematic experiments with existing state-of-the-art methods are presented to demonstrate its benefit. The experiments show that the proposed approach is competitive across a range of problems, including both deceptive and non-deceptive problems

Figures (6)

• Figure 1: The difference between standard mapping (left), and the bilevel mapping of interest in this study (right). On the left, commonly used surrogate modeling has one-to-one/many-to-one mapping between inputs and outputs, where a vector ${\mathbf{x}}$ corresponds to one value of $y$. On the right, the mapping involves a set of vectors (the LL PS $\mathbf{x}_{l}^{i*}$) corresponding to each given candidate UL vector $\mathbf{x}_u^i$, resulting in a one-to-many mapping scenario.
• Figure 2: The data for all available $\mathbf{x}_{u}$ and their corresponding LL PS is processed in this manner and then stacked together as one collective dataset provided to the NN for training.
• Figure 3: Proof-of-concept results for DS2 problem
• Figure 4: Flowchart of the proposed PSP-BLEMO framework
• Figure 5: Median result convergence plots. IGD values are shown in log scale.