Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SCaLE: Switching Cost aware Learning and Exploration

Neelkamal Bhuyan, Debankur Mukherjee, Adam Wierman

TL;DR

This paper tackles the problem of unbounded metric movement costs in bandit online convex optimization with high-dimensional dynamic quadratic hitting costs. It introduces SCaLE, an explore-then-exploit algorithm that learns an unknown curvature matrix $A$ from zeroth-order feedback via trace-norm matrix recovery and uses a spectral regret framework to separate eigenvalue and eigenbasis perturbations, achieving sublinear dynamic regret with rank-deficient and full-rank $A$. A lower bound shows that an exploration–exploitation trade-off is intrinsic, and HySCaLE offers a practical hybrid exploiting exploitation feedback in light-tailed environments. Theoretical results are complemented by extensive experiments demonstrating robustness to heavy tails and improved performance across regimes, with motivating applications in data-center thermal management and wake steering in wind farms. The work bridges online learning, control, and bandit optimization under movement costs, providing distribution-agnostic guarantees and actionable insights for large-scale, real-time decision making.

Abstract

This work addresses the fundamental problem of unbounded metric movement costs in bandit online convex optimization, by considering high-dimensional dynamic quadratic hitting costs and $\ell_2$-norm switching costs in a noisy bandit feedback model. For a general class of stochastic environments, we provide the first algorithm SCaLE that provably achieves a distribution-agnostic sub-linear dynamic regret, without the knowledge of hitting cost structure. En-route, we present a novel spectral regret analysis that separately quantifies eigenvalue-error driven regret and eigenbasis-perturbation driven regret. Extensive numerical experiments, against online-learning baselines, corroborate our claims, and highlight statistical consistency of our algorithm.

SCaLE: Switching Cost aware Learning and Exploration

TL;DR

This paper tackles the problem of unbounded metric movement costs in bandit online convex optimization with high-dimensional dynamic quadratic hitting costs. It introduces SCaLE, an explore-then-exploit algorithm that learns an unknown curvature matrix from zeroth-order feedback via trace-norm matrix recovery and uses a spectral regret framework to separate eigenvalue and eigenbasis perturbations, achieving sublinear dynamic regret with rank-deficient and full-rank . A lower bound shows that an exploration–exploitation trade-off is intrinsic, and HySCaLE offers a practical hybrid exploiting exploitation feedback in light-tailed environments. Theoretical results are complemented by extensive experiments demonstrating robustness to heavy tails and improved performance across regimes, with motivating applications in data-center thermal management and wake steering in wind farms. The work bridges online learning, control, and bandit optimization under movement costs, providing distribution-agnostic guarantees and actionable insights for large-scale, real-time decision making.

Abstract

This work addresses the fundamental problem of unbounded metric movement costs in bandit online convex optimization, by considering high-dimensional dynamic quadratic hitting costs and -norm switching costs in a noisy bandit feedback model. For a general class of stochastic environments, we provide the first algorithm SCaLE that provably achieves a distribution-agnostic sub-linear dynamic regret, without the knowledge of hitting cost structure. En-route, we present a novel spectral regret analysis that separately quantifies eigenvalue-error driven regret and eigenbasis-perturbation driven regret. Extensive numerical experiments, against online-learning baselines, corroborate our claims, and highlight statistical consistency of our algorithm.
Paper Structure (51 sections, 7 theorems, 100 equations, 22 figures, 3 algorithms)

This paper contains 51 sections, 7 theorems, 100 equations, 22 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. (i) A Fundamental Trade-off.
  3. (ii) Degeneracy in High Dimensions.
  4. Model and Preliminaries
  5. Algorithms and Regret Guarantees
  6. The SCaLE Algorithm
  7. Regret Lower Bound
  8. HySCaLE: Heuristic Adaptive Exploitation
  9. Numerical Experiments
  10. Experimental Setup
  11. Environment Generation.
  12. Algorithms and Baselines.
  13. Hyperparameter and Solver Settings.
  14. Case Studies
  15. Primary Evaluation.
  16. ...and 36 more sections

Key Result

Theorem 1

Dynamic regret of SCaLE$(\bar{\eta},r,\underline{\sigma^A_{r}})$, for $A$ with rank $r$, is bounded as: with high probability ($1-\exp(-C_0 m)$), where $C_0,c_1$ are universal constants from matrix estimation theory 7101247CaiZhang15, and $\Sigma$ is the upper limit on the covariance of the disturbance process $\{v_t-v_{t-1}\}_t$, that is, $\Sigma_{v_t-v_{t-1}} \preceq \Sigma$. The terms $\alpha_

Figures (22)

  • Figure 1: SCaLE & HySCaLE performance (this work); FTM: Smoothed online optimization benchmark zhang2021revisiting; POL/OAL: noisy/perfect online learning benchmark
  • Figure 2: Rank-deficiency hinders estimating curvature
  • Figure 3: ${\text{Regret}}_T$, ${\text{Regret}}_{1000}(t)$ and $\frac{\Delta {\text{Regret}}_{1000}(t)}{\Delta t}$, for $r=d=4, \sigma_r^A = 10^{-2}, \sigma_v = 50,\bar{\eta} = 10$ and $c_1$ set to $3$
  • Figure 4: $v_t-v_{t-1} \sim$ Cauchy distribution with $\sigma_v = 1$. Left plots for $\sigma_r^A = 10^{-2}$ and right plots for $\sigma_r^A = 1$. Test settings: $r=1$, $d=4$, $\bar{\eta} = 50$, $c=10$.
  • Figure 5: SCaLE and HySCaLE for $r=1, d=4.$ Left plots for $\sigma_r^A = 10^{-2}$ and right plots for $\sigma_r^A = 1.$ Test settings: $\bar{\eta} = 50$,$v_t-v_{t-1}\sim \mathcal{N}(0,50\cdot I)$, $c=10$
  • ...and 17 more figures

Theorems & Definitions (12)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • Remark 1
  • Theorem 2
  • Remark 2
  • Theorem 3
  • Proposition 1
  • Theorem 4
  • ...and 2 more