Decentralized Parameter-Free Online Learning

TL;DR

This work addresses the practical problem of hyperparameter tuning in decentralized online learning by introducing DECO, a pair of parameter-free algorithms that achieve sublinear network regret. By marrying the comparator-adaptive coin-betting framework with gossip-based consensus, DECO eliminates the need for learning-rate tuning while ensuring network-wide cooperation. Theoretical results show that the average local regret is topology-insensitive and that network regret decomposes into this local term plus a controllable disagreement term driven by information mixing, which can be reduced by increasing gossip rounds over time. Empirical evaluations on synthetic and real-world datasets demonstrate robust, tuning-free performance across network topologies and reveal a concrete trade-off between communication effort and learning performance, with practical implications for distributed sensing and collaborative machine learning.

Abstract

We propose the first parameter-free decentralized online learning algorithms with network regret guarantees, which achieve sublinear regret without requiring hyperparameter tuning. This family of algorithms connects multi-agent coin-betting and decentralized online learning via gossip steps. To enable our decentralized analysis, we introduce a novel "betting function" formulation for coin-betting that simplifies the multi-agent regret analysis. Our analysis shows sublinear network regret bounds and is validated through experiments on synthetic and real datasets. This family of algorithms is applicable to distributed sensing, decentralized optimization, and collaborative ML applications.

Figures (6)

• Figure 1: Operational loop for an agent using DECO-i. Key elements specific to this variant, related to tracking and communicating wealth, are highlighted in red.
• Figure 2: Operational loop for DECO-ii. The process is simplified by decoupling from wealth, requiring only the accumulated gradients to be maintained and gossiped. Differences from DECO-i are highlighted in green.
• Figure 3: Final cumulative network loss of DOGD as a function of its initial learning rate ($\eta_0$) on synthetic data, compared to the constant performance of our parameter-free DECO algorithms. The U-shaped curve for DOGD highlights its extreme sensitivity to hyperparameter choice, where a suboptimal setting leads to loss that is orders of magnitude worse. Our DECO algorithms achieve good performance without any tuning, regardless of the potential function used.
• Figure 4: Performance of DECO-ii (KT) on synthetic data for Erdős-Rényi random graphs with varying edge probability $p$. Top: Cumulative network loss. Cumulative loss growth improves with better connectivity, as faster information mixing reduces the disagreement among agents. Bottom: Per-round network loss (smoothed with a 250-round moving average). The smoothed loss shows that more connected networks not only accumulate less total loss but also converge faster to the centralized ideal's steady-state per-round loss.
• Figure 5: Cumulative network loss of DECO-ii (KT) on a cycle graph with different gossip schedules, $q(t)$. Increasing the communication budget from constant to logarithmic and linear progressively reduces loss, allowing the algorithm to approach the centralized oracle's performance. This highlights the explicit trade-off between communication cost and learning performance.

Theorems & Definitions (25)

• Definition 1: Subgradient
• Lemma 1: Reward-Regret Relationship mcmahan_unconstrained_2014
• Definition 2: Coin Betting Potential orabona_coin_2016
• Definition 3: Betting function
• Definition 4: Gossip Matrix
• Lemma 2: Gossip convergence rate boyd_randomized_2006
• Corollary 1: Gossip component-wise convergence rate
• Lemma 3: Wealth lower bound in multi-agent setting
• proof
• Remark 1