Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Safety in the Face of Adversity: Achieving Zero Constraint Violation in Online Learning with Slowly Changing Constraints

Bassel Hamoud, Ilnura Usmanova, Kfir Y. Levy

TL;DR

This work tackles safe online learning under dynamically evolving constraints by targeting zero constraint violation in every round while achieving sublinear dynamic regret. It introduces a primal-dual framework that performs online gradient ascent in the dual space, aided by a dichotomous step-size that adapts to safe and danger phases, ensuring safety even as constraints drift slowly by at most $\delta$ per round. A weaker optimization oracle is used to realize an efficient dual approach, with formal guarantees that sublinear primal regret of $\mathcal{O}(\sqrt{(V_{f,T}+V_{g,T})T})$ is attainable in the strongly convex setting, and convex extensions via the Allen-Zhu trick yielding additional regret bounds. The theoretical results rely on strong duality under Slater conditions, Lipschitz/smooth properties of $f_t$ and $g_t$, and the slowly changing constrains assumption, making the framework applicable to safety-critical, non-stationary environments. These findings establish the first provable zero-violation safety guarantees in online learning with slowly changing constraints and unify safety with sublinear regret in a dynamic OCO setting.

Abstract

We present the first theoretical guarantees for zero constraint violation in Online Convex Optimization (OCO) across all rounds, addressing dynamic constraint changes. Unlike existing approaches in constrained OCO, which allow for occasional safety breaches, we provide the first approach for maintaining strict safety under the assumption of gradually evolving constraints, namely the constraints change at most by a small amount between consecutive rounds. This is achieved through a primal-dual approach and Online Gradient Ascent in the dual space. We show that employing a dichotomous learning rate enables ensuring both safety, via zero constraint violation, and sublinear regret. Our framework marks a departure from previous work by providing the first provable guarantees for maintaining absolute safety in the face of changing constraints in OCO.

Safety in the Face of Adversity: Achieving Zero Constraint Violation in Online Learning with Slowly Changing Constraints

TL;DR

This work tackles safe online learning under dynamically evolving constraints by targeting zero constraint violation in every round while achieving sublinear dynamic regret. It introduces a primal-dual framework that performs online gradient ascent in the dual space, aided by a dichotomous step-size that adapts to safe and danger phases, ensuring safety even as constraints drift slowly by at most per round. A weaker optimization oracle is used to realize an efficient dual approach, with formal guarantees that sublinear primal regret of is attainable in the strongly convex setting, and convex extensions via the Allen-Zhu trick yielding additional regret bounds. The theoretical results rely on strong duality under Slater conditions, Lipschitz/smooth properties of and , and the slowly changing constrains assumption, making the framework applicable to safety-critical, non-stationary environments. These findings establish the first provable zero-violation safety guarantees in online learning with slowly changing constraints and unify safety with sublinear regret in a dynamic OCO setting.

Abstract

We present the first theoretical guarantees for zero constraint violation in Online Convex Optimization (OCO) across all rounds, addressing dynamic constraint changes. Unlike existing approaches in constrained OCO, which allow for occasional safety breaches, we provide the first approach for maintaining strict safety under the assumption of gradually evolving constraints, namely the constraints change at most by a small amount between consecutive rounds. This is achieved through a primal-dual approach and Online Gradient Ascent in the dual space. We show that employing a dichotomous learning rate enables ensuring both safety, via zero constraint violation, and sublinear regret. Our framework marks a departure from previous work by providing the first provable guarantees for maintaining absolute safety in the face of changing constraints in OCO.
Paper Structure (40 sections, 16 theorems, 105 equations)

This paper contains 40 sections, 16 theorems, 105 equations.

Table of Contents

  1. INTRODUCTION
  2. Related Work.
  3. PROBLEM STATEMENT
  4. Notation and Definitions.
  5. Assumptions.
  6. Preliminaries.
  7. OUR APPROACH
  8. Warm Up
  9. A More Efficient Approach
  10. Weak Optimization Oracle and Dual Regret.
  11. The Main Idea.
  12. Safety in Online Learning with Changing Constraints
  13. The Main Principle.
  14. Safety Guarantees
  15. Bounding the Regret
  16. ...and 25 more sections

Key Result

Lemma 1

Under Assumptions assum:bounded_set-assum:obj_smooth_strconv_lipsc and assum:non_shallow_constr_and_strong_duality, the optimal dual values $\lambda_t^* = \arg\max_{\lambda \geq 0} d_t(\lambda)$ and $\tilde{\lambda}_t^* = \arg\max_{\lambda \geq 0} \tilde{d}_t(\lambda)$ are bounded by $\hat{\lambda}

Theorems & Definitions (33)

  • Lemma 1
  • Lemma 2
  • Theorem 1
  • Lemma 3
  • proof
  • Corollary 1
  • Theorem 2
  • Lemma 4
  • Lemma 5
  • Corollary 2
  • ...and 23 more