Revisiting Projection-Free Online Learning with Time-Varying Constraints

TL;DR

This work addresses projection-free constrained online convex optimization with time-varying constraints by constructing a Lyapunov-based composite surrogate loss and applying a parameter-free online Frank-Wolfe (OFW) extension. It delivers improved regret and cumulative constraint violation bounds for both general convex and strongly convex losses, and extends the framework to bandit feedback using one-point gradient estimators with comparable rates. The proposed OFW-TVC and SCOFW-TVC algorithms, together with their bandit counterparts, achieve $\mathrm{Regret}_T=\mathcal{O}(T^{3/4})$ and $Q_T=\mathcal{O}(T^{3/4}\log T)$ in the general convex full-information setting, and $\mathrm{Regret}_T=\mathcal{O}(T^{2/3})$ and $Q_T=\mathcal{O}(T^{5/6})$ in the strongly convex regime, with corresponding bandit results including $\mathcal{O}(T^{3/4})$ and $\mathcal{O}(T^{2/3}\log T)$ terms. Experiments on real datasets like MovieLens and FilmTrust validate the theoretical gains, demonstrating the practicality of projection-free COCO under time-varying constraints. The work advances projection-free online learning by eliminating projection costs while rigorously balancing regret and long-term constraint satisfaction.

Abstract

We investigate constrained online convex optimization, in which decisions must belong to a fixed and typically complicated domain, and are required to approximately satisfy additional time-varying constraints over the long term. In this setting, the commonly used projection operations are often computationally expensive or even intractable. To avoid the time-consuming operation, several projection-free methods have been proposed with an $\mathcal{O}(T^{3/4} \sqrt{\log T})$ regret bound and an $\mathcal{O}(T^{7/8})$ cumulative constraint violation (CCV) bound for general convex losses. In this paper, we improve this result and further establish \textit{novel} regret and CCV bounds when loss functions are strongly convex. The primary idea is to first construct a composite surrogate loss, involving the original loss and constraint functions, by utilizing the Lyapunov-based technique. Then, we propose a parameter-free variant of the classical projection-free method, namely online Frank-Wolfe (OFW), and run this new extension over the online-generated surrogate loss. Theoretically, for general convex losses, we achieve an $\mathcal{O}(T^{3/4})$ regret bound and an $\mathcal{O}(T^{3/4} \log T)$ CCV bound, both of which are order-wise tighter than existing results. For strongly convex losses, we establish new guarantees of an $\mathcal{O}(T^{2/3})$ regret bound and an $\mathcal{O}(T^{5/6})$ CCV bound. Moreover, we also extend our methods to a more challenging setting with bandit feedback, obtaining similar theoretical findings. Empirically, experiments on real-world datasets have demonstrated the effectiveness of our methods.

Figures (2)

• Figure 1: Experimental results on the MovieLens dataset.
• Figure 2: Experimental results on the Film Trust dataset.

Theorems & Definitions (16)

• Definition 1
• Definition 2
• Lemma 1
• Theorem 1
• Theorem 2
• Lemma 2
• Theorem 3
• Theorem 4
• Lemma 3
• Lemma 4