Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Projection-free Algorithms for Online Convex Optimization with Adversarial Constraints

Dhruv Sarkar, Aprameyo Chakrabartty, Subhamon Supantha, Palash Dey, Abhishek Sinha

TL;DR

This work extends Online Convex Optimization to a constrained, adversarial setting where constraint functions are revealed online, and it eliminates the expensive projection step by using a projection-free approach based on a surrogate cost fed to an adaptive Online Conditional Gradient (OCG) method. The core strategy combines a Lyapunov-drift analysis with surrogate costs of the form $\hat{f}_t(x)= V f_t(x) + \Phi'(Q(t)) g_t^+(x)$, and relies on a single LP solve per round to move along a feasible direction. The authors establish tilde-$O(T^{3/4})$ bounds for both regret and cumulative constraint violations (CCV) in both full-information and bandit settings, improving over prior projection-free COCO methods. The framework is demonstrated on an online shortest path problem with time-varying constraints, showing favorable performance and practical efficiency, and it lays groundwork for adaptive, projection-free online learning in structured decision spaces.

Abstract

We study a generalization of the Online Convex Optimization (OCO) framework with time-varying adversarial constraints. In this problem, after selecting a feasible action from the convex decision set $X,$ a convex constraint function is revealed alongside the cost function in each round. Our goal is to design a computationally efficient learning policy that achieves a small regret with respect to the cost functions and a small cumulative constraint violation (CCV) with respect to the constraint functions over a horizon of length $T$. It is well-known that the projection step constitutes the major computational bottleneck of the standard OCO algorithms. However, for many structured decision sets, linear functions can be efficiently optimized over the decision set. We propose a *projection-free* online policy which makes a single call to a Linear Program (LP) solver per round. Our method outperforms state-of-the-art projection-free online algorithms with adversarial constraints, achieving improved bounds of $\tilde{O}(T^{\frac{3}{4}})$ for both regret and CCV. The proposed algorithm is conceptually simple - it first constructs a surrogate cost function as a non-negative linear combination of the cost and constraint functions. Then, it passes the surrogate costs to a new, adaptive version of the online conditional gradient subroutine, which we propose in this paper.

Projection-free Algorithms for Online Convex Optimization with Adversarial Constraints

TL;DR

This work extends Online Convex Optimization to a constrained, adversarial setting where constraint functions are revealed online, and it eliminates the expensive projection step by using a projection-free approach based on a surrogate cost fed to an adaptive Online Conditional Gradient (OCG) method. The core strategy combines a Lyapunov-drift analysis with surrogate costs of the form , and relies on a single LP solve per round to move along a feasible direction. The authors establish tilde- bounds for both regret and cumulative constraint violations (CCV) in both full-information and bandit settings, improving over prior projection-free COCO methods. The framework is demonstrated on an online shortest path problem with time-varying constraints, showing favorable performance and practical efficiency, and it lays groundwork for adaptive, projection-free online learning in structured decision spaces.

Abstract

We study a generalization of the Online Convex Optimization (OCO) framework with time-varying adversarial constraints. In this problem, after selecting a feasible action from the convex decision set a convex constraint function is revealed alongside the cost function in each round. Our goal is to design a computationally efficient learning policy that achieves a small regret with respect to the cost functions and a small cumulative constraint violation (CCV) with respect to the constraint functions over a horizon of length . It is well-known that the projection step constitutes the major computational bottleneck of the standard OCO algorithms. However, for many structured decision sets, linear functions can be efficiently optimized over the decision set. We propose a *projection-free* online policy which makes a single call to a Linear Program (LP) solver per round. Our method outperforms state-of-the-art projection-free online algorithms with adversarial constraints, achieving improved bounds of for both regret and CCV. The proposed algorithm is conceptually simple - it first constructs a surrogate cost function as a non-negative linear combination of the cost and constraint functions. Then, it passes the surrogate costs to a new, adaptive version of the online conditional gradient subroutine, which we propose in this paper.

Paper Structure

This paper contains 41 sections, 14 theorems, 105 equations, 7 figures, 1 table, 6 algorithms.

Table of Contents

  1. Introduction
  2. Our contributions
  3. Preliminaries: Adaptive OCO Algorithms
  4. Adaptive Online Conditional Gradient Algorithm
  5. Bounding term $(A)$:
  6. Bounding term (B):
  7. Projection-free COCO
  8. Algorithm Design and Analysis
  9. The Regret Decomposition Inequality
  10. Regret and CCV Bound
  11. Analysis:
  12. Projection-Free CBCO
  13. Experiments: Online Shortest Path Problem with Constraints
  14. Setup:
  15. Dataset:
  16. ...and 26 more sections

Key Result

Theorem 2.1

Consider a sequence of convex cost functions $\{\hat{f}_t\}_{t \geq 1},$ where the function $\hat{f}_t$ is $L_t$-Lipschitz such that the Lipschitz constants $L_t$'s are monotone non-decreasing, i.e.,$L_t\leq L_{t+1}, \forall t\geq 1$. Then, the Online Conditional Gradient algorithm with adaptive lea In particular, if $\max_{1\leq t \leq T} \frac{L_{t}}{L_{t-1}}$ is upper bounded by a constant, the

Figures (7)

  • Figure 1: Comparing CCV achieved by our policy, Algorithm 1 of sinha2024optimal, and Algorithm 3 of garber2024projectionfree
  • Figure 2: Regret achieved by our policy, Algorithm 1 of sinha2024optimal, and Algorithm 3 of garber2024projectionfree
  • Figure 3: Comparison of execution times of the proposed Algorithm \ref{['our_alg']} and Algorithm 3 of garber2024projectionfree. The run-time for sinha2024optimal's algorithm is two orders of magnitude larger (more than $1000$s) and hence, is not shown on the figure.
  • Figure 4: Variation of CCV for our policy, Algorithm 1 of sinha2024optimal, and Algorithm 3 and 4 by garber2024projectionfree
  • Figure 5: Variation of Regret for our policy, Algorithm 1 of sinha2024optimal, and Algorithm 3 and 4 by garber2024projectionfree
  • ...and 2 more figures

Theorems & Definitions (22)

  • Theorem 2.1
  • proof
  • Lemma 2.2
  • Lemma 2.3
  • Theorem 2.4
  • Theorem 3.4
  • Lemma 3.5
  • Lemma 3.6
  • Remark 3.1
  • Remark 3.2
  • ...and 12 more