Asynchronous Message-Passing and Zeroth-Order Optimization Based Distributed Learning with a Use-Case in Resource Allocation in Communication Networks

TL;DR

This work develops an asynchronous distributed learning framework where agents cooperatively optimize a global objective while maintaining individual parameter tasks, using zeroth-order (cost-only) oracles and scalar communications to preserve privacy and reduce bandwidth. The approach leverages randomized smoothing to construct a smooth surrogate and derives a gradient estimator from local zeroth-order queries, enabling update rules that tolerate bounded random delays in a message-passing network. The authors prove convergence for smooth nonconvex objectives and establish a $\mathcal{O}(1/\sqrt{t})$-type rate (modulated by the smoothing parameter $\mu$), including a discussion of the trade-offs when simultaneous two-point queries are constrained by practical sampling. The framework is instantiated in a deep-learning-based resource allocation use-case in wireless networks, where transmitters collaboratively train individual policies to maximize a global data-rate reward, demonstrating the method’s practicality for online, scalable distributed learning with privacy and communication efficiency benefits.

Abstract

Distributed learning and adaptation have received significant interest and found wide-ranging applications in machine learning and signal processing. While various approaches, such as shared-memory optimization, multi-task learning, and consensus-based learning (e.g., federated learning and learning over graphs), focus on optimizing either local costs or a global cost, there remains a need for further exploration of their interconnections. This paper specifically focuses on a scenario where agents collaborate towards a common task (i.e., optimizing a global cost equal to aggregated local costs) while effectively having distinct individual tasks (i.e., optimizing individual local parameters in a local cost). Each agent's actions can potentially impact other agents' performance through interactions. Notably, each agent has access to only its local zeroth-order oracle (i.e., cost function value) and shares scalar values, rather than gradient vectors, with other agents, leading to communication bandwidth efficiency and agent privacy. Agents employ zeroth-order optimization to update their parameters, and the asynchronous message-passing between them is subject to bounded but possibly random communication delays. This paper presents theoretical convergence analyses and establishes a convergence rate for nonconvex problems. Furthermore, it addresses the relevant use-case of deep learning-based resource allocation in communication networks and conducts numerical experiments in which agents, acting as transmitters, collaboratively train their individual policies to maximize a global reward, e.g., a sum of data rates.

Figures (4)

• Figure 1: An example of a message-passing architecture with asynchronous agents. The communication delays are possibly random and vary over time. The detailed workflow is provided in \ref{['alg:proposed']}.
• Figure 2: Convergence of the proposed asynchronous distributed learning approach with different values of maximum delay $D^{\rm max}$, defined by \ref{['eq:timestamp']} and \ref{['ass:buffer:quantity']}. There are $m=24$ agents, each communicating with $N$ neighboring agents, and participating in training with probability $p$. The comparison is against centralized learning with fully connected agents ($N=m$) in synchronous mode and with no communication delay ($p=1$).
• Figure 3: Convergence of the proposed asynchronous distributed learning approach with different batch sizes $B$ defined in \ref{['eq:barr']}. There are $m = 24$ agents, each communicating with $N=4$ neighboring agents, and participating in training with a probability of $p=0.9$. The duration of one iteration in (b) is set equal to the typical average channel coherence time of 25 ms.
• Figure 4: Convergence of the proposed asynchronous distributed learning approach with different numbers of agents $m$. Each agent communicates with $N$ neighboring agents and participates in training with probability $p$. The performance is normalized by the maximum achieved sum rate for each $m$. The comparison is against centralized learning with fully connected agents ($N=m$) in synchronous mode and with no communication delay ($p=1$).

Theorems & Definitions (6)

• proof
• proof
• proof
• proof
• proof
• proof