Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inverse Optimization via Learning Feasible Regions

Ke Ren, Peyman Mohajerin Esfahani, Angelos Georghiou

TL;DR

This paper addresses inverse optimization by shifting from learning objective functions to learning feasible regions encoded by a parametric constraint $g_{\\theta}$. It introduces two losses, predictability and suboptimality, and a rich hypothesis class that parameterizes feasible sets through a latent $\\bm z$ and a primitive set $\\mathcal{Z}$, enabling discontinuous decision policies. A smoothing-based block-coordinate descent algorithm is developed, along with exact convex and MILP reformulations for special cases, to train the model efficiently. Empirical results on synthetic data and two power-system problems demonstrate accurate recovery of feasible regions, superior performance to objective-learning baselines, and scalable learning of power-network structures. These contributions advance data-driven IO by providing tractable training tools for learning constraints and by validating the approach in practical energy system contexts.

Abstract

We study inverse optimization (IO), where the goal is to use a parametric optimization program as the hypothesis class to infer relationships between input-decision pairs. Most of the literature focuses on learning only the objective function, as learning the constraint function (i.e., feasible regions) leads to nonconvex training programs. Motivated by this, we focus on learning feasible regions for known linear objectives and introduce two training losses along with a hypothesis class to parameterize the constraint function. Our hypothesis class surpasses the previous objective-only method by naturally capturing discontinuous behaviors in input-decision pairs. We introduce a customized block coordinate descent algorithm with a smoothing technique to solve the training problems, while for further restricted hypothesis classes, we reformulate the training optimization as a tractable convex program or mixed integer linear program. Synthetic experiments and two power system applications, including comparisons with state-of-the-art approaches, showcase and validate the proposed approach.

Inverse Optimization via Learning Feasible Regions

TL;DR

This paper addresses inverse optimization by shifting from learning objective functions to learning feasible regions encoded by a parametric constraint . It introduces two losses, predictability and suboptimality, and a rich hypothesis class that parameterizes feasible sets through a latent and a primitive set , enabling discontinuous decision policies. A smoothing-based block-coordinate descent algorithm is developed, along with exact convex and MILP reformulations for special cases, to train the model efficiently. Empirical results on synthetic data and two power-system problems demonstrate accurate recovery of feasible regions, superior performance to objective-learning baselines, and scalable learning of power-network structures. These contributions advance data-driven IO by providing tractable training tools for learning constraints and by validating the approach in practical energy system contexts.

Abstract

We study inverse optimization (IO), where the goal is to use a parametric optimization program as the hypothesis class to infer relationships between input-decision pairs. Most of the literature focuses on learning only the objective function, as learning the constraint function (i.e., feasible regions) leads to nonconvex training programs. Motivated by this, we focus on learning feasible regions for known linear objectives and introduce two training losses along with a hypothesis class to parameterize the constraint function. Our hypothesis class surpasses the previous objective-only method by naturally capturing discontinuous behaviors in input-decision pairs. We introduce a customized block coordinate descent algorithm with a smoothing technique to solve the training problems, while for further restricted hypothesis classes, we reformulate the training optimization as a tractable convex program or mixed integer linear program. Synthetic experiments and two power system applications, including comparisons with state-of-the-art approaches, showcase and validate the proposed approach.

Paper Structure

This paper contains 27 sections, 6 theorems, 42 equations, 7 figures, 7 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Works
  4. Learning Frameworks
  5. Loss Function
  6. Measuring Performance
  7. Hypothesis Class for $g_{\theta}$ and Reformulations
  8. Reformulation for Linear Objective Functions
  9. Adaptive Smoothing
  10. Convex and Mixed-integer Reformulations
  11. Numerical Experiments
  12. Problem 1: Inferring Generation Policy without Knowing Constraints
  13. Problem 2: Learning Power Network Structures
  14. Limitations and Future Work
  15. Impact Statements
  16. ...and 12 more sections

Key Result

Proposition 2.1

For any $\bm \theta\in\Theta$ and the IO data pair $(\bm x,\bm s)$ generated by the forward model forward_problem, both predictability $\ell_{\bm\theta}^\text{p}(\bm x,\bm s)$ and suboptimality $\ell_{\bm\theta}^\text{sub}(\bm x,\bm s)$ loss defined in losses satisfy

Figures (7)

  • Figure 1: Pictorial representation of the predictability and suboptimality loss functions.
  • Figure 2: Top: Pictorial representation of true predictability and suboptimality losses. Bottom: relationship between estimated out-of-sample loss ($\ell_{\bm\theta}^{\text{p}},\,\ell_{\bm\theta}^{\text{sub}}$) and true out-of-sample loss ($\ell^{\text{p}},\,\ell^{\text{sub}}$).
  • Figure 3: Top: Power network topology used in the forward problem in Section \ref{['sec:real']}. Bottom: Recovered network from inverse optimization in Section \ref{['learningnetwork']}.
  • Figure 4: Training loss and test metrics. Blue: Training loss (predictability). Red: Out-of-sample predictability loss. Cyan: True predictability loss. Green: True suboptimality loss. Yellow: Out-of-sample suboptimality loss.
  • Figure 5: Comparison between convergence behaviors of predictability and suboptimality losses via both Algorithm \ref{['alg:vanilla']} and \ref{['Alg:smooth']}. The y-axis represents the achieved training loss. Each box plot is based on the same $10$ randomly generated datasets.
  • ...and 2 more figures

Theorems & Definitions (16)

  • Proposition 2.1: Full characterization
  • Remark 3.1: Choice of primitive sets
  • Example 3.2: Constraint vs. objective learning in IO
  • Theorem 3.3: Exact reformulation
  • Remark 3.4: Suboptimality loss as smoothed predictability loss
  • Definition 3.5: Smoothed predictability loss
  • Proposition 3.6: Convergence
  • Proposition 3.7: Tractable convex reformulation
  • proof
  • Lemma 2.1
  • ...and 6 more