Efficient Near-Optimal Algorithm for Online Shortest Paths in Directed Acyclic Graphs with Bandit Feedback Against Adaptive Adversaries

TL;DR

This work tackles online shortest path selection on directed acyclic graphs under bandit feedback against adaptive adversaries. It introduces a computationally efficient algorithm that achieves a near-minimax high-probability regret of $\tilde{O}(\sqrt{|E|T\log |\mathcal{X}|})$ by combining a novel loss estimator with centroid-based graph decompositions, along with an augmented path representation to handle unequal path lengths. The method reduces general graphs to a compact, equivalent DAG $G^\dagger$ with path length $O(\log |\mathcal{X}|)$, enabling efficient FTRL updates and implicit exploration. Beyond online shortest paths, the framework yields improved high-probability regret guarantees across multiple combinatorial domains, including hypercubes, multi-task MAB, extensive-form games, and related settings, and establishes near-optimal lower bounds in certain DAG classes.

Abstract

In this paper, we study the online shortest path problem in directed acyclic graphs (DAGs) under bandit feedback against an adaptive adversary. Given a DAG $G = (V, E)$ with a source node $v_{\mathsf{s}}$ and a sink node $v_{\mathsf{t}}$, let $X \subseteq \{0,1\}^{|E|}$ denote the set of all paths from $v_{\mathsf{s}}$ to $v_{\mathsf{t}}$. At each round $t$, we select a path $\mathbf{x}_t \in X$ and receive bandit feedback on our loss $\langle \mathbf{x}_t, \mathbf{y}_t \rangle \in [-1,1]$, where $\mathbf{y}_t$ is an adversarially chosen loss vector. Our goal is to minimize regret with respect to the best path in hindsight over $T$ rounds. We propose the first computationally efficient algorithm to achieve a near-minimax optimal regret bound of $\tilde O(\sqrt{|E|T\log |X|})$ with high probability against any adaptive adversary, where $\tilde O(\cdot)$ hides logarithmic factors in the number of edges $|E|$. Our algorithm leverages a novel loss estimator and a centroid-based decomposition in a nontrivial manner to attain this regret bound. As an application, we show that our algorithm for DAGs provides state-of-the-art efficient algorithms for $m$-sets, extensive-form games, the Colonel Blotto game, shortest walks in directed graphs, hypercubes, and multi-task multi-armed bandits, achieving improved high-probability regret guarantees in all these settings.

Figures (8)

• Figure 1: Example $G$ and $G^\dagger$ according to conversion in \ref{['sec:alg-centroid']}. The longest path from source to sink in $G^\dagger$ is upper bounded by $\calO(\log |\calX|)$. See \ref{['fig:enter-label']} in Appendix for more details.
• Figure 2: An example graph conversion from $G$ to $G^\dagger$ is shown. The non-tree edges $E \setminus E^\clubsuit$ are shaded in $G$, and they correspond to the shaded edges in $G^\dagger$. The graph $S = (V, E^\clubsuit)$ has a centroid vertex $D$. Removing $D$ from $S$ results in three subtrees: $S_A$, $S_G$, and $S_F$. The new linked edges for the corresponding centroids are shaded in the graphs on the right. Recall $C(\cdot)$ is the number of distinct path from source to the vertex.
• Figure 3: Conversion of hypercube to DAG
• Figure 4: Conversion of Multi-task MAB to DAG
• Figure 5: Example extensive-form game 1

Theorems & Definitions (51)

• Lemma 2
• proof
• Lemma 3
• proof
• proof
• Lemma 4: slivkins2019introduction
• Lemma 5: Chain Rule
• proof
• Lemma 6: fiegel2023adapting
• Corollary 2: fiegel2023adapting