Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Survey

TL;DR

This survey addresses Resource Allocation Optimization (RAO) in dynamic, decentralized environments by surveying Multi-Agent Reinforcement Learning (MARL) approaches. It maps MARL fundamentals (MDPs, Dec-POMDPs, CTCE/DTDE/CTDE paradigms) to RAO challenges, and categorizes MARL algorithms across diverse application domains including telecom, energy, distributed computing, transportation, and manufacturing. Key contributions include a structured taxonomy of RAO resources, a review of classical RAO methods and their limitations, and a synthesis of MARL solutions, benchmarks, and future directions (e.g., hierarchical MARL, mean-field, graph-based MARL, and credit assignment schemes). The survey highlights the practical impact of MARL in enabling scalable, adaptable, and coordinated resource management, while outlining still-present challenges such as non-stationarity, scalability, heterogeneity, and safe exploration that motivate ongoing research.

Abstract

Multi-Agent Reinforcement Learning (MARL) has become a powerful framework for numerous real-world applications, modeling distributed decision-making and learning from interactions with complex environments. Resource Allocation Optimization (RAO) benefits significantly from MARL's ability to tackle dynamic and decentralized contexts. MARL-based approaches are increasingly applied to RAO challenges across sectors playing pivotal roles to Industry 4.0 developments. This survey provides a comprehensive review of recent MARL algorithms for RAO, encompassing core concepts, classifications, and a structured taxonomy. By outlining the current research landscape and identifying primary challenges and future directions, this survey aims to support researchers and practitioners in leveraging MARL's potential to advance resource allocation solutions.

Figures (8)

• Figure 1: MARL solution for RAO. Any resources can be allocated by several agents to complete tasks or activities. Each agents has its own policy to handle resource allocation determined by the rewards and observations of the whole system states that may come from resources and tasks situations.
• Figure 2: Complete manuscript body structure used in this survey: Preliminary section mainly covers the fundamental of RL and MARL methods. Then, we introduce the concept of RAO, its classical solution and highlight the challenges in the recent trends which can be solved using MARL algorithm and discussed in the RAO leveraging MARL section.
• Figure 3: Resource classification by divisibility, availability, and location properties.
• Figure 4: Comparison of Training and Execution Paradigms in MARL: (a) CTCE is a fully centralized framework which combines all observations and actions of the agents into a joint observation-action space. (b) DTDE is a term for fully decentralized setting that treats all agents independently with their own observation, action, and reward. (c) CTDE framework has training in a centralized manner with information from other agent, then deploy the trained policy to each agent independently.
• Figure 5: Earth Observation Mission Environment in BSK-RL: The number of satellite can be defined either single satellite or multi-satellite. Each satellite have each actions: 1) Downlinking: Transmit collected data to the ground station with predefined transmission speed and delete the data after successfully downlinked; 2) Imaging or Scanning: Satellite capturing image of a target using optical sensor or scanning any object on the earth surface using radar sensor (with two different payloads and can be used for different tasks); 3) Charging: The satellite is in charging mode and pointing its solar panel towards sun direction to maximize solar energy absorption; 4) Desaturating: There is a condition of the satellite's rotating wheel rotates saturatedly and the speed should be reduced to control their attitude.