Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derivative-Free Bilevel Optimization with Inexact Lower-Level Solutions

Edoardo Cesaroni, Giampaolo Liuzzi, Stefano Lucidi

Abstract

In this work, we propose derivative-free framework for bilevel optimization. We consider both the upper and lower-level problems with bound constraints on the variables, as well as general nonlinear constraints, assuming that first-order information (in the upper-level) is not available or it is impractical to obtain. The lower-level problem is solved with an accuracy that is progressively refined throughout the optimization process. We first analyze the case in which the upper-level problem is subject only to bound constraints, establishing convergence to Clarke-Jahn stationary points when the refinement process is allowed to reach its maximum precision. When a limitation is imposed on this refinement process, we prove convergence to approximate stationary points using an extended notion of Goldstein stationarity. Finally, we extend the proposed framework to handle more complex constraints via an exact penalty function approach, proving convergence to stationary points under suitable assumptions. A comprehensive numerical study on 160 problems from the BOLIB collection shows that the adaptive accuracy strategy consistently yields better results than fixed-precision solves, with its benefits becoming more pronounced as the required lower-level accuracy becomes more stringent.

Derivative-Free Bilevel Optimization with Inexact Lower-Level Solutions

Abstract

In this work, we propose derivative-free framework for bilevel optimization. We consider both the upper and lower-level problems with bound constraints on the variables, as well as general nonlinear constraints, assuming that first-order information (in the upper-level) is not available or it is impractical to obtain. The lower-level problem is solved with an accuracy that is progressively refined throughout the optimization process. We first analyze the case in which the upper-level problem is subject only to bound constraints, establishing convergence to Clarke-Jahn stationary points when the refinement process is allowed to reach its maximum precision. When a limitation is imposed on this refinement process, we prove convergence to approximate stationary points using an extended notion of Goldstein stationarity. Finally, we extend the proposed framework to handle more complex constraints via an exact penalty function approach, proving convergence to stationary points under suitable assumptions. A comprehensive numerical study on 160 problems from the BOLIB collection shows that the adaptive accuracy strategy consistently yields better results than fixed-precision solves, with its benefits becoming more pronounced as the required lower-level accuracy becomes more stringent.
Paper Structure (17 sections, 13 theorems, 104 equations, 2 figures, 2 tables, 2 algorithms)

This paper contains 17 sections, 13 theorems, 104 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Assumptions
  4. Notations and definitions
  5. Bound-Constrained Upper-Level Problem
  6. A Linesearch Derivative-free Method for Bilevel Programming
  7. Convergence analysis for the exact case ($\bar{\zeta} = 0$)
  8. Convergence analysis for the inexact case ($\bar{\zeta} > 0$)
  9. Handling of General Inequality Upper-Level Constraints
  10. Numerical Results
  11. Test problems collection
  12. Performance evaluation methodology
  13. Algorithms implementation details and parameter settings
  14. Effect of multiple step sizes
  15. Effect of Projected Extrapolation
  16. ...and 2 more sections

Key Result

Proposition 1

\newlabelprop:nocycle_projectedLS0 The Projected Extrapolation cannot cycle indefinitely between Step 11 and Step 17.

Figures (2)

  • Figure 1: Data profiles (top row) and performance profiles (bottom row) for $\bar{\zeta} = 10^{-6}$. Left half: MS-DFN-LLA (multiple step sizes) vs. DFN-LLA (single step size). Right half: MS-DFN-LLA (with extrapolation) vs. MS-DFN-NL-LLA (without). Odd columns: $\tau = 10^{-3}$; even columns: $\tau = 10^{-6}$.
  • Figure 2: Data profiles (columns 1--2) and performance profiles (columns 3--4) comparing MS-DFN-LLF (fixed accuracy) with MS-DFN-LLA (adaptive accuracy). Rows correspond to $\bar{\zeta} = 10^{-3}$ (top), $10^{-6}$ (middle), $10^{-9}$ (bottom). Odd columns: $\tau = 10^{-3}$; even columns: $\tau = 10^{-6}$.

Theorems & Definitions (22)

  • Remark 1
  • Remark 2
  • Definition 1: Clarke and Clarke-Jahn generalized directional derivative
  • Definition 2: Dense sequence
  • Definition 1: Cone of Feasible Directions
  • Definition 2: Clarke and Clarke-Jahn stationarity
  • Definition 3: $(\delta, \epsilon)$-Goldstein stationarity
  • Proposition 1
  • Proposition 2
  • Proposition 3
  • ...and 12 more