Beyond $\tilde{O}(\sqrt{T})$ Constraint Violation for Online Convex Optimization with Adversarial Constraints
Abhishek Sinha, Rahul Vaze
TL;DR
This work studies COCO, where an online learner must minimize both regret and cumulative constraint violation under adversarial constraints. By introducing a tunable parameter $\beta$ and leveraging adaptive small-loss regret bounds with Lyapunov surrogate costs, the authors achieve a trade-off: sublinear regret $\tilde{O}(\sqrt{dT}+T^{\beta})$ and CCV $\tilde{O}(dT^{1-\beta})$ in the convex setting, with sharper first-order guarantees under smoothness. The core strategy decomposes COCO into a Constrained Expert problem via a $\delta$-cover and applies an adaptive Hedge algorithm on surrogate losses, plus an online gradient-descent-based policy for the smooth case. These results imply substantial CCV reductions in safety-critical online optimization while maintaining sublinear regret, with practical implications for safety-critical systems and budget-constrained decision-making.
Abstract
We study Online Convex Optimization with adversarial constraints (COCO). At each round a learner selects an action from a convex decision set and then an adversary reveals a convex cost and a convex constraint function. The goal of the learner is to select a sequence of actions to minimize both regret and the cumulative constraint violation (CCV) over a horizon of length $T$. The best-known policy for this problem achieves $O(\sqrt{T})$ regret and $\tilde{O}(\sqrt{T})$ CCV. In this paper, we improve this by trading off regret to achieve substantially smaller CCV. This trade-off is especially important in safety-critical applications, where satisfying the safety constraints is non-negotiable. Specifically, for any bounded convex cost and constraint functions, we propose an online policy that achieves $\tilde{O}(\sqrt{dT}+ T^β)$ regret and $\tilde{O}(dT^{1-β})$ CCV, where $d$ is the dimension of the decision set and $β\in [0,1]$ is a tunable parameter. We begin with a special case, called the $\textsf{Constrained Expert}$ problem, where the decision set is a probability simplex and the cost and constraint functions are linear. Leveraging a new adaptive small-loss regret bound, we propose a computationally efficient policy for the $\textsf{Constrained Expert}$ problem, that attains $O(\sqrt{T\ln N}+T^β)$ regret and $\tilde{O}(T^{1-β} \ln N)$ CCV for $N$ number of experts. The original problem is then reduced to the $\textsf{Constrained Expert}$ problem via a covering argument. Finally, with an additional $M$-smoothness assumption, we propose a computationally efficient first-order policy attaining $O(\sqrt{MT}+T^β)$ regret and $\tilde{O}(MT^{1-β})$ CCV.