Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Review of Bilevel Optimization: Methods, Emerging Applications, and Recent Advancements

Dhaval Pujara, Ankur Sinha

TL;DR

This survey consolidates the theoretical foundations and algorithmic strategies for bilevel optimization, detailing optimistic and pessimistic formulations, and surveying classical (KKT, duality, value function, penalty, trust-region) and evolutionary methods. It also covers mixed-integer and multiobjective extensions, and situates these methods in real-world contexts such as tolling, interdiction, and NAS. Key contributions include the Bilevel Optimization based Decomposition (BOBD) framework and the formalization of NAS within a bilevel paradigm, illustrating practical automation of optimization and architecture search. The paper highlights ongoing challenges—particularly discrete and multiobjective bilevel problems—and argues for scalable, hybrid, and application-driven approaches moving forward.

Abstract

This paper presents a comprehensive review of techniques proposed in the literature for solving bilevel optimization problems encountered in various real-life applications. Bilevel optimization is an appropriate choice for hierarchical decision-making situations, where a decision-maker needs to consider a possible response from stakeholder(s) for each of its actions to achieve his own goals. Mathematically, it leads to a nested optimization structure, in which a primary (leader's) optimization problem contains a secondary (follower's) optimization problem as a constraint. Various forms of bilevel problems, including linear, mixed-integer, single-objective, and multi-objective, are covered. For bilevel problem solving methods, various classical and evolutionary approaches are explained. Along with an overview of various areas of applications, two recent considerations of bilevel approach are introduced. The first application involves a bilevel decomposition approach for solving general optimization problems, and the second application involves Neural Architecture Search (NAS), which is a prime example of a bilevel optimization problem in the area of machine learning.

A Review of Bilevel Optimization: Methods, Emerging Applications, and Recent Advancements

TL;DR

This survey consolidates the theoretical foundations and algorithmic strategies for bilevel optimization, detailing optimistic and pessimistic formulations, and surveying classical (KKT, duality, value function, penalty, trust-region) and evolutionary methods. It also covers mixed-integer and multiobjective extensions, and situates these methods in real-world contexts such as tolling, interdiction, and NAS. Key contributions include the Bilevel Optimization based Decomposition (BOBD) framework and the formalization of NAS within a bilevel paradigm, illustrating practical automation of optimization and architecture search. The paper highlights ongoing challenges—particularly discrete and multiobjective bilevel problems—and argues for scalable, hybrid, and application-driven approaches moving forward.

Abstract

This paper presents a comprehensive review of techniques proposed in the literature for solving bilevel optimization problems encountered in various real-life applications. Bilevel optimization is an appropriate choice for hierarchical decision-making situations, where a decision-maker needs to consider a possible response from stakeholder(s) for each of its actions to achieve his own goals. Mathematically, it leads to a nested optimization structure, in which a primary (leader's) optimization problem contains a secondary (follower's) optimization problem as a constraint. Various forms of bilevel problems, including linear, mixed-integer, single-objective, and multi-objective, are covered. For bilevel problem solving methods, various classical and evolutionary approaches are explained. Along with an overview of various areas of applications, two recent considerations of bilevel approach are introduced. The first application involves a bilevel decomposition approach for solving general optimization problems, and the second application involves Neural Architecture Search (NAS), which is a prime example of a bilevel optimization problem in the area of machine learning.

Paper Structure

This paper contains 18 sections, 2 theorems, 21 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Bilevel Formulation
  3. Optimistic Position
  4. Pessimistic Position
  5. Bilevel Problem Solving Methods
  6. Karush-Kuhn-Tucker Conditions based Single Level Reduction
  7. Duality-based Single Level Reduction
  8. Value Function based Single Level Reduction
  9. Descent Methods
  10. Penalty Function Approach
  11. Trust Region Approach
  12. Evolutionary Approach
  13. Mixed Integer Bilevel Optimization
  14. Multiobjective Bilevel Optimization
  15. Real-life Applications of Bilevel Optimization
  16. ...and 3 more sections

Key Result

Theorem 1

If the objective function and constraints of bilevel optimization problem ($F$, $f$, $G_p$, $g_q$) are sufficiently smooth, the constraint region $\Phi$ is non-empty and compact, and the Mangasarian-Fromowitz constraint qualification holds at all points, then the problem is guaranteed to have an opt

Figures (4)

  • Figure 1: A sketch of decision making mechanism in bilevel optimization problem
  • Figure 2: A network map depicting connections between research topics addressed using bilevel optimization
  • Figure 3: AutoOpt framework sinha_AutoOpt to automate the optimization problem-solving task
  • Figure 4: An outline of NAS: automated process for identifying the optimal architecture from the complex search space

Theorems & Definitions (6)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Theorem 2
  • Definition 3
  • Definition 4