A simple algorithm for the simple bilevel programming (SBP) problem

TL;DR

This work addresses the simple bilevel programming (SBP) problem, defined as $\min f(x)$ subject to $x \in \argmin\{ g(y) : y \in C \}$ with $C$ convex and compact and $f,g$ convex, allowing a smooth lower-level objective without requiring a Lipschitz gradient on $\nabla g$. It introduces the SBP-LFS algorithm, a projected-gradient–style method that avoids Lipschitz assumptions on the upper level and uses a sequence of relaxed lower-level problems $(P_k)$ with $\alpha_k \to \alpha = \min_C g$ to drive convergence; the method computes a projected gradient step, adaptively determines $\alpha_k$ via an inner search, and solves an $\varepsilon$-minimization of $f$ over $\{ g(x) \le \alpha_k + \eta_k \}$, with convergence to SBP solutions under mild conditions. The paper provides convergence analysis showing that accumulation points of the iterates are SBP solutions and discusses how compactness can be relaxed if $\arg\min_C g$ exists. Numerical experiments in MATLAB with CVX illustrate SBP-LFS on problems including an affine-upper-level SBP, sparse linear systems, distance-to-solution-set estimation, and a Markowitz portfolio model, demonstrating robustness, competitiveness, and practical applicability.

Abstract

In this article we intend to develop a simple and implementable algorithm for minimizing a convex function over the solution set of another convex optimization problem. Such a problem is often referred to as a simple bilevel programming (SBP) problem. One of the key features of our algorithm is that we make no assumption on the diferentiability of the upper level objective, though we will assume that the lower level objective is smooth. Another key feature of the algorithm is that it does not assume that the lower level objective has a Lipschitz gradient, which is a standard assumption in most of the well-known algorithms for this class of problems. We present the convergence analysis and also some numerical experiments demonstrating the efectiveness of the algorithm.

Theorems & Definitions (16)

• Example 1.1
• Example 2.1
• Theorem 2.1
• Remark 2.1
• Remark 2.2
• Lemma 3.1
• Lemma 3.2
• Lemma 3.3
• Lemma 3.4
• proof