Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Prescriptive PCA: Dimensionality Reduction for Two-stage Stochastic Optimization

Long He, Ho-Yin Mak

TL;DR

The paper tackles preserving downstream optimality when learning a low-dimensional representation for a two-stage stochastic program with recourse. It introduces prescriptive PCA (PPCA), which casts dimensionality reduction as a rank-constrained, distributionally-robust covariance split solved via semidefinite programming to minimize an upper bound on downstream suboptimality. The approach is validated on synthetic warehouse/transshipment and a NYC taxi pre-allocation case, where PPCA consistently yields smaller optimality gaps at low dimensions than standard PCA and reduces computational burden. This work provides a principled framework for aligning unsupervised dimensionality reduction with prescriptive optimization, with potential extensions to sparse or supervised settings and broader prescriptive tasks.

Abstract

In this paper, we consider the alignment between an upstream dimensionality reduction task of learning a low-dimensional representation of a set of high-dimensional data and a downstream optimization task of solving a stochastic program parameterized by said representation. In this case, standard dimensionality reduction methods (e.g., principal component analysis) may not perform well, as they aim to maximize the amount of information retained in the representation and do not generally reflect the importance of such information in the downstream optimization problem. To address this problem, we develop a prescriptive dimensionality reduction framework that aims to minimize the degree of suboptimality in the optimization phase. For the case where the downstream stochastic optimization problem has an expected value objective, we show that prescriptive dimensionality reduction can be performed via solving a distributionally-robust optimization problem, which admits a semidefinite programming relaxation. Computational experiments based on a warehouse transshipment problem and a vehicle repositioning problem show that our approach significantly outperforms principal component analysis with real and synthetic data sets.

Prescriptive PCA: Dimensionality Reduction for Two-stage Stochastic Optimization

TL;DR

The paper tackles preserving downstream optimality when learning a low-dimensional representation for a two-stage stochastic program with recourse. It introduces prescriptive PCA (PPCA), which casts dimensionality reduction as a rank-constrained, distributionally-robust covariance split solved via semidefinite programming to minimize an upper bound on downstream suboptimality. The approach is validated on synthetic warehouse/transshipment and a NYC taxi pre-allocation case, where PPCA consistently yields smaller optimality gaps at low dimensions than standard PCA and reduces computational burden. This work provides a principled framework for aligning unsupervised dimensionality reduction with prescriptive optimization, with potential extensions to sparse or supervised settings and broader prescriptive tasks.

Abstract

In this paper, we consider the alignment between an upstream dimensionality reduction task of learning a low-dimensional representation of a set of high-dimensional data and a downstream optimization task of solving a stochastic program parameterized by said representation. In this case, standard dimensionality reduction methods (e.g., principal component analysis) may not perform well, as they aim to maximize the amount of information retained in the representation and do not generally reflect the importance of such information in the downstream optimization problem. To address this problem, we develop a prescriptive dimensionality reduction framework that aims to minimize the degree of suboptimality in the optimization phase. For the case where the downstream stochastic optimization problem has an expected value objective, we show that prescriptive dimensionality reduction can be performed via solving a distributionally-robust optimization problem, which admits a semidefinite programming relaxation. Computational experiments based on a warehouse transshipment problem and a vehicle repositioning problem show that our approach significantly outperforms principal component analysis with real and synthetic data sets.
Paper Structure (16 sections, 8 theorems, 43 equations, 5 figures, 1 algorithm)

This paper contains 16 sections, 8 theorems, 43 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Dimensionality Reduction for Stochastic Optimization
  4. The Limitation of PCA
  5. Prescriptive PCA
  6. Distributionally-Robust Bound
  7. Tight Distributionally Robust Bound
  8. Computational Experiments with Synthetic Data
  9. Problem Setting
  10. Synthetic Data Generation
  11. Performance Evaluation
  12. Case Study: NYC Taxi Pre-Allocation
  13. Conclusion
  14. Proofs
  15. Proof of Proposition \ref{['prop:PPCA_SDP']}
  16. ...and 1 more sections

Key Result

Proposition 1

Suppose the linear program eq:2nd_stage has complete recourse. Let $\bm{z}_0$ be a realization of $\tilde{\bm{z}}_0$. Then, for any $\bm{z}_1, \bm{z}_e$ such that $\bm{z}_1 + \bm{z}_e = \bm{z}_0$, it holds that:

Figures (5)

  • Figure 1: Computational time for solving stochastic program using LDR against the number of dimensions ($k$) when $\zeta_i\sim$Normal(2,1).
  • Figure 2: $\zeta_i \sim$Normal(2,1) without noise.
  • Figure 3: $\zeta_i \sim$Normal(2,1) with noise.
  • Figure 4: The optimality gaps using PCA and PPCA for NYC Taxi Pre-allocation.
  • Figure 5: The top two principal components (PCs) from PPCA and PCA.

Theorems & Definitions (13)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • Proposition 4
  • Theorem 1
  • Lemma 1
  • proof
  • Lemma 2
  • ...and 3 more