Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

End-to-End Conformal Calibration for Optimization Under Uncertainty

Christopher Yeh, Nicolas Christianson, Alan Wu, Adam Wierman, Yisong Yue

TL;DR

The paper tackles decision-making under uncertainty by learning calibrated uncertainty sets for conditional robust optimization (CRO) in an end-to-end framework. It integrates conformal prediction to guaranteeCoverage while enabling end-to-end training through the downstream loss, using PICNNs to represent general convex uncertainty sets and to maintain tractable CRO reformulations. The authors derive exact gradients through the conformal calibration step, provide tractable CRO reformulations for box and ellipsoidal sets, and extend to general convex sets via PICNNs, with theoretical guarantees and practical validation on battery storage arbitrage and portfolio optimization. Empirically, end-to-end conformally calibrated CRO consistently improves downstream task performance over estimate-then-optimize baselines, demonstrating the value of calibrated, task-aware uncertainty in robust decision-making.

Abstract

Machine learning can significantly improve performance for decision-making under uncertainty in a wide range of domains. However, ensuring robustness guarantees requires well-calibrated uncertainty estimates, which can be difficult to achieve in high-capacity prediction models such as deep neural networks. Moreover, in high-dimensional settings, there may be many valid uncertainty estimates, each with their own performance profile - i.e., not all uncertainty is equally valuable for downstream decision-making. To address this problem, this paper develops an end-to-end framework to learn the uncertainty estimates for conditional robust optimization, with robustness and calibration guarantees provided by conformal prediction. In addition, we propose to represent arbitrary convex uncertainty sets with partially input-convex neural networks, which are learned as part of our framework. Our approach consistently improves upon two-stage estimate-then-optimize baselines on concrete applications in energy storage arbitrage and portfolio optimization.

End-to-End Conformal Calibration for Optimization Under Uncertainty

TL;DR

The paper tackles decision-making under uncertainty by learning calibrated uncertainty sets for conditional robust optimization (CRO) in an end-to-end framework. It integrates conformal prediction to guaranteeCoverage while enabling end-to-end training through the downstream loss, using PICNNs to represent general convex uncertainty sets and to maintain tractable CRO reformulations. The authors derive exact gradients through the conformal calibration step, provide tractable CRO reformulations for box and ellipsoidal sets, and extend to general convex sets via PICNNs, with theoretical guarantees and practical validation on battery storage arbitrage and portfolio optimization. Empirically, end-to-end conformally calibrated CRO consistently improves downstream task performance over estimate-then-optimize baselines, demonstrating the value of calibrated, task-aware uncertainty in robust decision-making.

Abstract

Machine learning can significantly improve performance for decision-making under uncertainty in a wide range of domains. However, ensuring robustness guarantees requires well-calibrated uncertainty estimates, which can be difficult to achieve in high-capacity prediction models such as deep neural networks. Moreover, in high-dimensional settings, there may be many valid uncertainty estimates, each with their own performance profile - i.e., not all uncertainty is equally valuable for downstream decision-making. To address this problem, this paper develops an end-to-end framework to learn the uncertainty estimates for conditional robust optimization, with robustness and calibration guarantees provided by conformal prediction. In addition, we propose to represent arbitrary convex uncertainty sets with partially input-convex neural networks, which are learned as part of our framework. Our approach consistently improves upon two-stage estimate-then-optimize baselines on concrete applications in energy storage arbitrage and portfolio optimization.
Paper Structure (42 sections, 4 theorems, 43 equations, 4 figures, 1 algorithm)

This paper contains 42 sections, 4 theorems, 43 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Statement and Background
  3. Comparison to related work.
  4. End-to-End Training of Conformally Calibrated Uncertainty Sets
  5. Assumptions.
  6. Conformal uncertainty set calibration
  7. Representations of the uncertainty set
  8. Box uncertainty sets.
  9. Ellipsoidal uncertainty sets.
  10. End-to-end training and calibration
  11. Representing General Convex Uncertainty Sets via PICNNs
  12. Experiments
  13. Problem descriptions
  14. Price forecasting for battery storage.
  15. Portfolio optimization
  16. ...and 27 more sections

Key Result

Lemma 1

Let $D_\text{cal} = \{(x_i,y_i)\}_{i=1}^M$ be a calibration dataset drawn exchangeably (e.g., i.i.d.) from $\mathcal{P}$, and let $s_i = s_\theta(x_i, y_i)$. If $q = \Call{Quantile}{\{s_i\}_{i=1}^M,\ 1-\alpha}$ (see alg:dauq) is the empirical $(1-\alpha)$-quantile of the set $\{s_i\}_{i=1}^M$ and $(

Figures (4)

  • Figure 1: Our proposed framework for end-to-end conformal calibration for optimization under uncertainty updates the machine learning model using gradients from the task loss.
  • Figure 2: Task loss performance (mean $\pm$1 stddev across 10 runs) for the battery storage problem with no distribution shift. Lower values are better.
  • Figure 4: Task loss performance (mean $\pm$1 stddev across 10 runs) for the battery storage problem with distribution shift. Lower values are better.
  • Figure 6: Task loss performance (mean $\pm$1 stddev across 10 runs) for the portfolio optimization problem on synthetic data. Lower values are better.

Theorems & Definitions (6)

  • Definition 1: marginal coverage
  • Lemma 1: from angelopoulos_conformal_2023, Appendix D
  • Proposition 1
  • Theorem 1
  • Theorem 2
  • proof