FAuNO: Semi-Asynchronous Federated Reinforcement Learning Framework for Task Offloading in Edge Systems

TL;DR

Experiments show that FAuNO consistently matches or exceeds heuristic and federated multi-agent RL baselines in reducing task loss and latency, underscoring its adaptability to dynamic edge-computing scenarios.

Abstract

Edge computing addresses the growing data demands of connected-device networks by placing computational resources closer to end users through decentralized infrastructures. This decentralization challenges traditional, fully centralized orchestration, which suffers from latency and resource bottlenecks. We present \textbf{FAuNO} -- \emph{Federated Asynchronous Network Orchestrator} -- a buffered, asynchronous \emph{federated reinforcement-learning} (FRL) framework for decentralized task offloading in edge systems. FAuNO adopts an actor-critic architecture in which local actors learn node-specific dynamics and peer interactions, while a federated critic aggregates experience across agents to encourage efficient cooperation and improve overall system performance. Experiments in the \emph{PeersimGym} environment show that FAuNO consistently matches or exceeds heuristic and federated multi-agent RL baselines in reducing task loss and latency, underscoring its adaptability to dynamic edge-computing scenarios.

Figures (8)

• Figure 1: Edge System Architecture of our system model. The workers are capable of independently offloading tasks, exchanging information, and FL model updates through the communication channels.
• Figure 2: Agents independently train local actor-critic models. Following algo. \ref{['algo:localtrain']}
• Figure 3: Agents asynchronously transmit their critic updates to the Global Manager, which stores them in a buffer.
• Figure 4: When a new update from an already known agent arrives, the older entry in the buffer is replaced, and the agent’s weight in the upcoming aggregation is increased according to its training steps.
• Figure 5: Once updates from $K$ distinct agents are available, these are aggregated following eq. \ref{['eq:aligning']}, producing a new global critic that is then redistributed to all agents.