Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained Online Convex Optimization with Memory and Predictions

Mohammed Abdullah, George Iosifidis, Salah Eddine Elayoubi, Tijani Chahed

Abstract

We study Constrained Online Convex Optimization with Memory (COCO-M), where both the loss and the constraints depend on a finite window of past decisions made by the learner. This setting extends the previously studied unconstrained online optimization with memory framework and captures practical problems such as the control of constrained dynamical systems and scheduling with reconfiguration budgets. For this problem, we propose the first algorithms that achieve sublinear regret and sublinear cumulative constraint violation under time-varying constraints, both with and without predictions of future loss and constraint functions. Without predictions, we introduce an adaptive penalty approach that guarantees sublinear regret and constraint violation. When short-horizon and potentially unreliable predictions are available, we reinterpret the problem as online learning with delayed feedback and design an optimistic algorithm whose performance improves as prediction accuracy improves, while remaining robust when predictions are inaccurate. Our results bridge the gap between classical constrained online convex optimization and memory-dependent settings, and provide a versatile learning toolbox with diverse applications.

Constrained Online Convex Optimization with Memory and Predictions

Abstract

We study Constrained Online Convex Optimization with Memory (COCO-M), where both the loss and the constraints depend on a finite window of past decisions made by the learner. This setting extends the previously studied unconstrained online optimization with memory framework and captures practical problems such as the control of constrained dynamical systems and scheduling with reconfiguration budgets. For this problem, we propose the first algorithms that achieve sublinear regret and sublinear cumulative constraint violation under time-varying constraints, both with and without predictions of future loss and constraint functions. Without predictions, we introduce an adaptive penalty approach that guarantees sublinear regret and constraint violation. When short-horizon and potentially unreliable predictions are available, we reinterpret the problem as online learning with delayed feedback and design an optimistic algorithm whose performance improves as prediction accuracy improves, while remaining robust when predictions are inaccurate. Our results bridge the gap between classical constrained online convex optimization and memory-dependent settings, and provide a versatile learning toolbox with diverse applications.
Paper Structure (30 sections, 10 theorems, 112 equations, 4 figures, 1 table, 3 algorithms)

This paper contains 30 sections, 10 theorems, 112 equations, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. COCO Bounds.
  4. COCO-M and Non-stochastic Control.
  5. Optimism.
  6. Summary.
  7. Preliminaries
  8. Background.
  9. Learning Model & Assumptions.
  10. Algorithm Design & Analysis
  11. Problem $\texttt{COCO-M}^2$: Double Memory Effect
  12. Discussion.
  13. Problem $\texttt{COCO-M}$
  14. Discussion
  15. Benefiting from Predictions
  16. ...and 15 more sections

Key Result

Lemma 1

Under Assumptions 1-3, performing OGD on eq:surrogate with step $\eta_t\!=\!\frac{\sqrt{2}\|\mathcal{X}\|}{2\sqrt{\sum_{\tau=1}^{t} \|\nabla \hat{\mathcal{L}}_t(x_t)\|^2}}$, we obtain: where the benchmark set depends on the problem: $\mathcal{X}_T^{\mathrm{b}}=\mathcal{X}_T^{m}$ for COCO-M$^2$ and $\mathcal{X}_T^{\mathrm{b}}=\mathcal{X}_T$ for COCO-M.

Figures (4)

  • Figure 1: decision stages of COCO-M (with predictions).
  • Figure 2: Diagonal: $Z_t(x_t)$ depends only on $x_t$, but includes loss/constraint components from next $m$ rounds. Vertical: $\mathcal{L}_t(x_{t-m}^t)$ includes function components only from round $t$, but depends on all past $m$ decisions.
  • Figure 3: Notation used throughout the paper
  • Figure 4: Performance of Algorithm \ref{['algo:memory-coco']} on $\texttt{COCO‑M}^{2}$ with $x^{\star}\!\in\!\mathcal{X}^{\text{m}}_T$.

Theorems & Definitions (18)

  • Remark 1
  • Lemma 1
  • Lemma 2: Regret decomposition sinha2024optimal
  • Theorem 1: Regret and CCV for $\texttt{COCO-M}^2$
  • Theorem 2: Regret and CCV with memory-free constraint
  • Lemma 3
  • Theorem 3
  • proof : Proof Theorem \ref{['thm:obj-memory']}
  • proof : Proof of Theorem \ref{['thm:full-memory']}
  • Theorem 4: Regret and CCV with memory-free constraint
  • ...and 8 more