Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Smart Predict-then-Optimize Method with Dependent Data: Risk Bounds and Calibration of Autoregression

Jixian Liu, Tao Xu, Jianping He, Chongrong Fang

Abstract

The predict-then-optimize (PTO) framework is indispensable for addressing practical stochastic decision-making tasks. It consists of two crucial steps: initially predicting unknown parameters of an optimization model and subsequently solving the problem based on these predictions. Elmachtoub and Grigas [1] introduced the Smart Predict-then-Optimize (SPO) loss for the framework, which gauges the decision error arising from predicted parameters, and a convex surrogate, the SPO+ loss, which incorporates the underlying structure of the optimization model. The consistency of these different loss functions is guaranteed under the assumption of i.i.d. training data. Nevertheless, various types of data are often dependent, such as power load fluctuations over time. This dependent nature can lead to diminished model performance in testing or real-world applications. Motivated to make intelligent predictions for time series data, we present an autoregressive SPO method directly targeting the optimization problem at the decision stage in this paper, where the conditions of consistency are no longer met. Therefore, we first analyze the generalization bounds of the SPO loss within our autoregressive model. Subsequently, the uniform calibration results in Liu and Grigas [2] are extended in the proposed model. Finally, we conduct experiments to empirically demonstrate the effectiveness of the SPO+ surrogate compared to the absolute loss and the least squares loss, especially when the cost vectors are determined by stationary dynamical systems and demonstrate the relationship between normalized regret and mixing coefficients.

Smart Predict-then-Optimize Method with Dependent Data: Risk Bounds and Calibration of Autoregression

Abstract

The predict-then-optimize (PTO) framework is indispensable for addressing practical stochastic decision-making tasks. It consists of two crucial steps: initially predicting unknown parameters of an optimization model and subsequently solving the problem based on these predictions. Elmachtoub and Grigas [1] introduced the Smart Predict-then-Optimize (SPO) loss for the framework, which gauges the decision error arising from predicted parameters, and a convex surrogate, the SPO+ loss, which incorporates the underlying structure of the optimization model. The consistency of these different loss functions is guaranteed under the assumption of i.i.d. training data. Nevertheless, various types of data are often dependent, such as power load fluctuations over time. This dependent nature can lead to diminished model performance in testing or real-world applications. Motivated to make intelligent predictions for time series data, we present an autoregressive SPO method directly targeting the optimization problem at the decision stage in this paper, where the conditions of consistency are no longer met. Therefore, we first analyze the generalization bounds of the SPO loss within our autoregressive model. Subsequently, the uniform calibration results in Liu and Grigas [2] are extended in the proposed model. Finally, we conduct experiments to empirically demonstrate the effectiveness of the SPO+ surrogate compared to the absolute loss and the least squares loss, especially when the cost vectors are determined by stationary dynamical systems and demonstrate the relationship between normalized regret and mixing coefficients.

Paper Structure

This paper contains 17 sections, 9 theorems, 33 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Formulation
  3. "Smart Predict, then Optimize" Framework
  4. Uniform Calibration
  5. Stationary $\beta$-mixing Processes
  6. Problem Formulation
  7. Proposed Algorithm
  8. Generalization Bounds and Calibration Results
  9. Generalization Bounds for the Dependent Data
  10. Calibration with Different Feasible Regions
  11. Computational Experiments
  12. Conclusions and Future Directions
  13. Redemancher Complexity and the Generalization Bounds
  14. Calibration Results
  15. Calibration between SPO+ and SPO Loss with i.i.d. data liu2021risk
  16. ...and 2 more sections

Key Result

Corollary 1

(McDonald, Shalizi and Schervish mcdonald2017nonparametric) For the $f \in \mathcal{H}$ has fixed memory l, we have that $f(y_{1:j}) = f(y_{j-l+1:j})$. Thus we have $R_{SPO, n}(f) = R_{SPO, l}(f)$.

Figures (3)

  • Figure 1: SPO-based Autoregression Model
  • Figure 2: Normalized test set SPO loss for the SPO+, least squares, and absolute loss methods on Knapsack instances.
  • Figure 3: Boxplot of the test set's normalized SPO loss for the SPO+ loss methods with enhancing mixing.

Theorems & Definitions (18)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Corollary 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • proof
  • Lemma 1
  • ...and 8 more