Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simultaneous Learning and Optimization via Misspecified Saddle Point Problems

Mohammad Mahdi Ahmadi, Erfan Yazdandoost Hamedani

TL;DR

This work tackles misspecified saddle point problems where an unknown parameter $\theta^*$, learned from data, influences the optimization objective. It develops two APD-based algorithms—Naive APD and Learning-aware APD—where the latter explicitly accounts for parameter dynamics and employs a backtracking step-size strategy, achieving a convergence rate of $\mathcal{O}(\log K / K)$ with a smaller constant. The paper further extends the framework to settings with multiple learning solutions, obtaining a $\mathcal{O}(1/\sqrt{K})$ rate under a structured, pessimistic formulation, and validates the approach on a misspecified portfolio problem, showing superior empirical performance. Collectively, these results provide a flexible, data-driven method for jointly solving optimization and learning tasks under parameter misspecification with practical impact on portfolio optimization and robust learning applications.

Abstract

We study a class of misspecified saddle point (SP) problems, where the optimization objective depends on an unknown parameter that must be learned concurrently from data. Unlike existing studies that assume parameters are fully known or pre-estimated, our framework integrates optimization and learning into a unified formulation, enabling a more flexible problem class. To address this setting, we propose two algorithms based on the accelerated primal-dual (APD) by Hamedani & Aybat 2021. In particular, we first analyze the naive extension of the APD method by directly substituting the evolving parameter estimates into the primal-dual updates; then, we design a new learning-aware variant of the APD method that explicitly accounts for parameter dynamics by adjusting the momentum updates. Both methods achieve a provable convergence rate of $\mathcal{O}(\log K / K)$, while the learning-aware approach attains a tighter $\mathcal{O}(1)$ constant and further benefits from an adaptive step-size selection enabled by a backtracking strategy. Furthermore, we extend the framework to problems where the learning problem admits multiple optimal solutions, showing that our modified algorithm for a structured setting achieves an $\mathcal{O}(1/\sqrt{K})$ rate. To demonstrate practical impact, we evaluate our methods on a misspecified portfolio optimization problem and show superior empirical performance compared to state-of-the-art algorithms.

Simultaneous Learning and Optimization via Misspecified Saddle Point Problems

TL;DR

This work tackles misspecified saddle point problems where an unknown parameter , learned from data, influences the optimization objective. It develops two APD-based algorithms—Naive APD and Learning-aware APD—where the latter explicitly accounts for parameter dynamics and employs a backtracking step-size strategy, achieving a convergence rate of with a smaller constant. The paper further extends the framework to settings with multiple learning solutions, obtaining a rate under a structured, pessimistic formulation, and validates the approach on a misspecified portfolio problem, showing superior empirical performance. Collectively, these results provide a flexible, data-driven method for jointly solving optimization and learning tasks under parameter misspecification with practical impact on portfolio optimization and robust learning applications.

Abstract

We study a class of misspecified saddle point (SP) problems, where the optimization objective depends on an unknown parameter that must be learned concurrently from data. Unlike existing studies that assume parameters are fully known or pre-estimated, our framework integrates optimization and learning into a unified formulation, enabling a more flexible problem class. To address this setting, we propose two algorithms based on the accelerated primal-dual (APD) by Hamedani & Aybat 2021. In particular, we first analyze the naive extension of the APD method by directly substituting the evolving parameter estimates into the primal-dual updates; then, we design a new learning-aware variant of the APD method that explicitly accounts for parameter dynamics by adjusting the momentum updates. Both methods achieve a provable convergence rate of , while the learning-aware approach attains a tighter constant and further benefits from an adaptive step-size selection enabled by a backtracking strategy. Furthermore, we extend the framework to problems where the learning problem admits multiple optimal solutions, showing that our modified algorithm for a structured setting achieves an rate. To demonstrate practical impact, we evaluate our methods on a misspecified portfolio optimization problem and show superior empirical performance compared to state-of-the-art algorithms.

Paper Structure

This paper contains 20 sections, 11 theorems, 109 equations, 1 figure, 4 algorithms.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Stochastic Optimization.
  4. Robust Optimization.
  5. Misspecified Problems and Data-driven Approaches.
  6. Contribution
  7. Portfolio Optimization Problem
  8. Robust Optimization with Anomaly-aware Learning
  9. Preliminaries
  10. Assumptions and Definitions
  11. Assumptions and Definitions
  12. Proposed Methods
  13. Naive Approach: Inexact Accelerated Primal-dual Method
  14. Learning-aware Accelerated Primal-dual Method
  15. Multiple Learning Solutions
  16. ...and 5 more sections

Key Result

Theorem 3.1

Suppose Assumption assump:phi-lip holds. Let $z \triangleq [x^\top, y^\top]^\top$. If $\{x_k,y_k,\theta_k\}_{k\geq 0}$ is generated by Algorithm alg:naive-mis-sp, using a constant parameter sequence $\{\tau_k,\sigma_k,\eta_k\}_{k\geq 0}$ satisfying Assumption assump:step-size-cond2. Then for any con where $\bar{z}_K \triangleq [\bar{x}_K^\top,\bar{y}_K^\top]^\top$ denotes the weighted average of t

Figures (1)

  • Figure 1: Comparison of our proposed Naive (blue) and Learning-aware (red) with IPALM (yellow) and VI (purple) across three datasets and three metrics over iteration $k$. From top to bottom: NASDAQ-100, Dow Jones, and Synthetic datasets. From left to right, the figures correspond to suboptimality, infeasibility, and learning solution metrics.

Theorems & Definitions (34)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Remark 3.1: Backtracking Step-Size Scheme for Learning
  • Theorem 3.1
  • proof
  • Remark 3.2
  • Remark 3.3: Convergence of Learning Sequence
  • Corollary 3.2
  • proof
  • ...and 24 more