Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Algorithms for Online Convex Optimization with Adversarial Constraints

Abhishek Sinha, Rahul Vaze

TL;DR

For the first time, it is shown that a simple first-order policy can simultaneously achieve regret and cumulative constraint violation in COCO, and in the case of strongly convex cost and convex constraint functions, the regret guarantee can be improved to $O(\log T)$ while keeping the CCV bound the same as above.

Abstract

A well-studied generalization of the standard online convex optimization (OCO) framework is constrained online convex optimization (COCO). In COCO, on every round, a convex cost function and a convex constraint function are revealed to the learner after it chooses the action for that round. The objective is to design an online learning policy that simultaneously achieves a small regret while ensuring a small cumulative constraint violation (CCV) against an adaptive adversary interacting over a horizon of length $T$. A long-standing open question in COCO is whether an online policy can simultaneously achieve $O(\sqrt{T})$ regret and $\tilde{O}(\sqrt{T})$ CCV without any restrictive assumptions. For the first time, we answer this in the affirmative and show that a simple first-order policy can simultaneously achieve these bounds. Furthermore, in the case of strongly convex cost and convex constraint functions, the regret guarantee can be improved to $O(\log T)$ while keeping the CCV bound the same as above. We establish these results by effectively combining adaptive OCO policies as a blackbox with Lyapunov optimization - a classic tool from control theory. Surprisingly, the analysis is short and elegant.

Optimal Algorithms for Online Convex Optimization with Adversarial Constraints

TL;DR

For the first time, it is shown that a simple first-order policy can simultaneously achieve regret and cumulative constraint violation in COCO, and in the case of strongly convex cost and convex constraint functions, the regret guarantee can be improved to while keeping the CCV bound the same as above.

Abstract

A well-studied generalization of the standard online convex optimization (OCO) framework is constrained online convex optimization (COCO). In COCO, on every round, a convex cost function and a convex constraint function are revealed to the learner after it chooses the action for that round. The objective is to design an online learning policy that simultaneously achieves a small regret while ensuring a small cumulative constraint violation (CCV) against an adaptive adversary interacting over a horizon of length . A long-standing open question in COCO is whether an online policy can simultaneously achieve regret and CCV without any restrictive assumptions. For the first time, we answer this in the affirmative and show that a simple first-order policy can simultaneously achieve these bounds. Furthermore, in the case of strongly convex cost and convex constraint functions, the regret guarantee can be improved to while keeping the CCV bound the same as above. We establish these results by effectively combining adaptive OCO policies as a blackbox with Lyapunov optimization - a classic tool from control theory. Surprisingly, the analysis is short and elegant.
Paper Structure (58 sections, 6 theorems, 74 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 58 sections, 6 theorems, 74 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Unconstrained OCO:
  4. Our Contributions
  5. The Constrained OCO (COCO) Problem
  6. Assumptions
  7. Remarks:
  8. Online Policy for COCO
  9. Design and Analysis of the Algorithm
  10. Defining the Surrogate Cost Functions
  11. The Regret Decomposition Inequality
  12. Convex Cost and Convex Constraint Functions
  13. $\blacksquare$ Performance Analysis
  14. An exponential Lyapunov function:
  15. Bounding the Regret:
  16. ...and 43 more sections

Key Result

Theorem 1

For the COCO problem with adversarially chosen $G$-Lipschitz cost and constraint functions, Algorithm coco_alg, with $\beta=(2GD)^{-1}, V=1, \Phi(x)= \exp(\frac{x}{2\sqrt{T}})-1,$ yields the following Regret and CCV bounds for any horizon length $T \geq 1:$ In the above, $D$ denotes the Euclidean diameter of the closed and convex admissible set $\mathcal{X}$.

Figures (3)

  • Figure 1: ROC curve obtained by varying $\lambda$
  • Figure 2: Typical variation of the CCV with time
  • Figure 3: A schematic for the online multi-task learning problem

Theorems & Definitions (7)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Definition 1
  • Theorem 4
  • Theorem 5
  • Theorem 6