Parameter-free Algorithms for the Stochastically Extended Adversarial Model

TL;DR

This work introduces the Stochastically Extended Adversarial (SEA) model to bridge adversarial and stochastic online convex optimization and develops parameter-free algorithms based on Optimistic Online Newton Step (OONS). By removing dependence on unknown problem parameters, the authors present comparator-adaptive (CA-OONS) and comparator- and Lipschitz-adaptive (CLA-OONS) variants that achieve regret bounds in terms of variance and variation quantities $\sigma_{1:T}^2$ and $\Sigma_{1:T}^2$, rather than the horizon $T$. CA-OONS handles unknown domain diameter with known Lipschitz constant, while CLA-OONS extends to fully unknown $D$ and $G$ using a multi-layer meta-framework and gradient clipping, yielding robust performance in the SEA model. The results advance practical parameter-free online learning in SEA and motivate future work on tighter $\|u\|_2$-dependence, gradient-query efficiency, and high-probability regret guarantees.

Abstract

We develop the first parameter-free algorithms for the Stochastically Extended Adversarial (SEA) model, a framework that bridges adversarial and stochastic online convex optimization. Existing approaches for the SEA model require prior knowledge of problem-specific parameters, such as the diameter of the domain $D$ and the Lipschitz constant of the loss functions $G$, which limits their practical applicability. Addressing this, we develop parameter-free methods by leveraging the Optimistic Online Newton Step (OONS) algorithm to eliminate the need for these parameters. We first establish a comparator-adaptive algorithm for the scenario with unknown domain diameter but known Lipschitz constant, achieving an expected regret bound of $\tilde{O}\big(\|u\|_2^2 + \|u\|_2(\sqrt{σ^2_{1:T}} + \sqrt{Σ^2_{1:T}})\big)$, where $u$ is the comparator vector and $σ^2_{1:T}$ and $Σ^2_{1:T}$ represent the cumulative stochastic variance and cumulative adversarial variation, respectively. We then extend this to the more general setting where both $D$ and $G$ are unknown, attaining the comparator- and Lipschitz-adaptive algorithm. Notably, the regret bound exhibits the same dependence on $σ^2_{1:T}$ and $Σ^2_{1:T}$, demonstrating the efficacy of our proposed methods even when both parameters are unknown in the SEA model.

Theorems & Definitions (27)

• Theorem 3.1
• Theorem 3.2
• Remark 3.3
• Theorem 4.1
• Remark 4.2: Dependency on $\|u\|_2^2$
• Remark 4.3: Adaptivity between gradient-variation bound and worst-case bound
• Remark 4.4: On dependence on the time horizon
• Theorem 4.5
• Remark 4.6: Discussion and challenges
• Proposition A.1: Linear separation between $\sigma_{1:T}^2$ and $\widetilde{\sigma}_{1:T}^2$