Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Overfitting in Adaptive Robust Optimization

Karl Zhu, Dimitris Bertsimas

TL;DR

Adaptive robust optimization (ARO) can be brittle because adaptivity makes some originally uncertainty-free constraints depend on realized uncertainty, potentially causing infeasibilities outside the modeled set. The authors propose constraint-dependent uncertainty sets and robust counterparts (RCs) to enforce probabilistic guarantees for softer constraints and deterministic guarantees for hard constraints, revealing that RCs act as regularization on the adaptive coefficients $V$ and shrink or permit adaptivity depending on the strength of the guarantees. They derive probabilistic RCs under Gaussian and distribution-free settings for ellipsoidal, ball-box, and budget uncertainty sets, as well as deterministic RCs for bounded and semi-bounded supports, and interpret these guarantees through a regularization lens that links guarantee strength to reduced adaptivity. The work provides principled guidance for designing uncertainty sets that balance robustness and adaptivity in ARO, with implications for energy systems and other domains where uncertainty is high but not uniformly critical across constraints.

Abstract

Adaptive robust optimization (ARO) extends static robust optimization by allowing decisions to depend on the realized uncertainty - weakly dominating static solutions within the modeled uncertainty set. However, ARO makes previous constraints that were independent of uncertainty now dependent, making it vulnerable to additional infeasibilities when realizations fall outside the uncertainty set. This phenomenon of adaptive policies being brittle is analogous to overfitting in machine learning. To mitigate against this, we propose assigning constraint-specific uncertainty set sizes, with harder constraints given stronger probabilistic guarantees. Interpreted through the overfitting lens, this acts as regularization: tighter guarantees shrink adaptive coefficients to ensure stability, while looser ones preserve useful flexibility. This view motivates a principled approach to designing uncertainty sets that balances robustness and adaptivity.

Overfitting in Adaptive Robust Optimization

TL;DR

Adaptive robust optimization (ARO) can be brittle because adaptivity makes some originally uncertainty-free constraints depend on realized uncertainty, potentially causing infeasibilities outside the modeled set. The authors propose constraint-dependent uncertainty sets and robust counterparts (RCs) to enforce probabilistic guarantees for softer constraints and deterministic guarantees for hard constraints, revealing that RCs act as regularization on the adaptive coefficients and shrink or permit adaptivity depending on the strength of the guarantees. They derive probabilistic RCs under Gaussian and distribution-free settings for ellipsoidal, ball-box, and budget uncertainty sets, as well as deterministic RCs for bounded and semi-bounded supports, and interpret these guarantees through a regularization lens that links guarantee strength to reduced adaptivity. The work provides principled guidance for designing uncertainty sets that balance robustness and adaptivity in ARO, with implications for energy systems and other domains where uncertainty is high but not uniformly critical across constraints.

Abstract

Adaptive robust optimization (ARO) extends static robust optimization by allowing decisions to depend on the realized uncertainty - weakly dominating static solutions within the modeled uncertainty set. However, ARO makes previous constraints that were independent of uncertainty now dependent, making it vulnerable to additional infeasibilities when realizations fall outside the uncertainty set. This phenomenon of adaptive policies being brittle is analogous to overfitting in machine learning. To mitigate against this, we propose assigning constraint-specific uncertainty set sizes, with harder constraints given stronger probabilistic guarantees. Interpreted through the overfitting lens, this acts as regularization: tighter guarantees shrink adaptive coefficients to ensure stability, while looser ones preserve useful flexibility. This view motivates a principled approach to designing uncertainty sets that balances robustness and adaptivity.

Paper Structure

This paper contains 7 sections, 2 theorems, 8 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Brittleness of Adaptive Policies
  3. Probabilistic and Deterministic Guarantees
  4. Probabilistic guarantees.
  5. Deterministic guarantees.
  6. A Regularization Perspective
  7. Discussion and Conclusion

Key Result

Proposition 1

Suppose $u_k$ has support $[L_k,U_k] \ \forall k \in [p]$, and $\boldsymbol{x}(\boldsymbol{u})$ is affinely adaptive (eq:adr). Then the following RC satisfies $\boldsymbol{a}_i^\top \boldsymbol{x}(\boldsymbol{u})\le b_i + \boldsymbol{d}_i^\top \boldsymbol{u}$ w.p. 1: where $\boldsymbol{L} = (L_k)_{k=1}^p, \boldsymbol{U} = (U_k)_{k=1}^p$ and $\boldsymbol{\alpha}, \boldsymbol{\beta} \in \mathbb{R}^

Figures (1)

  • Figure 1: Ellipsoidal guarantees: maximum feasible adaptivity $\|\boldsymbol{V}^\top \boldsymbol{a}\|_2$ as a function of the probabilistic guarantee $1-\varepsilon$. Under Gaussian assumptions, the underlying probabilistic bound is tighter, yielding less conservative feasibility regions. The distribution-free guarantee is looser, leading to more conservative regularization. In both cases, higher guarantees correspond to stronger regularization, shrinking adaptive flexibility and preventing brittle solutions.

Theorems & Definitions (2)

  • Proposition 1: Bounded support
  • Corollary 1: Semi-bounded support