On the Universal Near Optimality of Hedge in Combinatorial Settings

TL;DR

This work analyzes Hedge in full-information combinatorial online learning, establishing a universal lower bound that Hedge is near-optimal up to a $\sqrt{\log d}$ factor. It shows Hedge is suboptimal on $m$-sets (where $\log d \le m \le \sqrt{d}$) but optimal for online multitask learning, and it ties minimax optimality to a dilated-entropy regularizer in DAGs. The Online Mirror Descent framework with this dilated entropy is shown to be iterate-equivalent to Hedge on DAGs, providing a near-optimal and efficient regularization approach for a broad class of combinatorial domains. Together, these results map the landscape of Hedge’s performance across key combinatorial settings and connect it to efficient regret-minimizing algorithms via OMD.

Abstract

In this paper, we study the classical Hedge algorithm in combinatorial settings. In each round, the learner selects a vector $\boldsymbol{x}_t$ from a set $X \subseteq \{0,1\}^d$, observes a full loss vector $\boldsymbol{y}_t \in \mathbb{R}^d$, and incurs a loss $\langle \boldsymbol{x}_t, \boldsymbol{y}_t \rangle \in [-1,1]$. This setting captures several important problems, including extensive-form games, resource allocation, $m$-sets, online multitask learning, and shortest-path problems on directed acyclic graphs (DAGs). It is well known that Hedge achieves a regret of $O\big(\sqrt{T \log |X|}\big)$ after $T$ rounds of interaction. In this paper, we ask whether Hedge is optimal across all combinatorial settings. To that end, we show that for any $X \subseteq \{0,1\}^d$, Hedge is near-optimal--specifically, up to a $\sqrt{\log d}$ factor--by establishing a lower bound of $Ω\big(\sqrt{T \log(|X|)/\log d}\big)$ that holds for any algorithm. We then identify a natural class of combinatorial sets--namely, $m$-sets with $\log d \leq m \leq \sqrt{d}$--for which this lower bound is tight, and for which Hedge is provably suboptimal by a factor of exactly $\sqrt{\log d}$. At the same time, we show that Hedge is optimal for online multitask learning, a generalization of the classical $K$-experts problem. Finally, we leverage the near-optimality of Hedge to establish the existence of a near-optimal regularizer for online shortest-path problems in DAGs--a setting that subsumes a broad range of combinatorial domains. Specifically, we show that the classical Online Mirror Descent (OMD) algorithm, when instantiated with the dilated entropy regularizer, is iterate-equivalent to Hedge, and therefore inherits its near-optimal regret guarantees for DAGs.

Figures (1)

• Figure 1: An overview of our results. The $x$-axis indexes different combinatorial decision sets $\mathcal{X} \subseteq \{0,1\}^d$, and the $y$-axis shows the optimal regret over $T$ rounds. We show that Hedge is near-optimal for all $\mathcal{X}$, up to a $\sqrt{\log d}$ factor. For $m$-sets, Hedge is provably suboptimal by a factor of $\sqrt{\log d}$, whereas in structured settings such as online multitask learning and important families of DAGs, it is in fact optimal.

Theorems & Definitions (22)

• Theorem 2.2: Regret Bound for OMD, rakhlin2013onlinesyrgkanis2015fast
• Lemma 3.1: Sauer–Shelah Lemma sauer1972densityshelah1972combinatorial
• Theorem 4.1
• Theorem \ref{thm:regret-lb-general}
• Proposition \ref{prop:msets-reward-set}
• Lemma \ref{lm:msets-convexity}
• Lemma \ref{lm:msets-range}
• Theorem \ref{thm:m-set-all-algo-lb}
• Theorem \ref{thm:m-set-hedge-lb}
• Theorem \ref{thm:multitask}