Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Training in Multi-Agent Reinforcement Learning: A Communication-Free Framework for the Box-Pushing Problem

David Ge, Hao Ji

TL;DR

This paper proposes a model called Shared Pool of Information (SPI), which enables information to be accessible to all agents and facilitates coordination, reducing force conflicts among agents and enhancing exploration efficiency.

Abstract

Self-organizing systems consist of autonomous agents that can perform complex tasks and adapt to dynamic environments without a central controller. Prior research often relies on reinforcement learning to enable agents to gain the skills needed for task completion, such as in the box-pushing environment. However, when agents push from opposing directions during exploration, they tend to exert equal and opposite forces on the box, resulting in minimal displacement and inefficient training. This paper proposes a model called Shared Pool of Information (SPI), which enables information to be accessible to all agents and facilitates coordination, reducing force conflicts among agents and enhancing exploration efficiency. Through computer simulations, we demonstrate that SPI not only expedites the training process but also requires fewer steps per episode, significantly improving the agents' collaborative effectiveness.

Efficient Training in Multi-Agent Reinforcement Learning: A Communication-Free Framework for the Box-Pushing Problem

TL;DR

This paper proposes a model called Shared Pool of Information (SPI), which enables information to be accessible to all agents and facilitates coordination, reducing force conflicts among agents and enhancing exploration efficiency.

Abstract

Self-organizing systems consist of autonomous agents that can perform complex tasks and adapt to dynamic environments without a central controller. Prior research often relies on reinforcement learning to enable agents to gain the skills needed for task completion, such as in the box-pushing environment. However, when agents push from opposing directions during exploration, they tend to exert equal and opposite forces on the box, resulting in minimal displacement and inefficient training. This paper proposes a model called Shared Pool of Information (SPI), which enables information to be accessible to all agents and facilitates coordination, reducing force conflicts among agents and enhancing exploration efficiency. Through computer simulations, we demonstrate that SPI not only expedites the training process but also requires fewer steps per episode, significantly improving the agents' collaborative effectiveness.

Paper Structure

This paper contains 31 sections, 19 equations, 16 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. The Box pushing problem
  4. Multiagent Communication
  5. Centralized vs Decentralized Learning
  6. Methodology
  7. Environment
  8. Action Space and State Space
  9. Reward
  10. Speed Factor
  11. Implicit Coordination Through Shared Pool of Information
  12. Proposed Shared Pool of Information
  13. Fitness Test using BMD
  14. Origin Avoidance
  15. Uniformity of Angular Spread
  16. ...and 16 more sections

Figures (16)

  • Figure 1: Illustration of the box environment.
  • Figure 2: Illustration of the action space.
  • Figure 3: Illustration of the state space.
  • Figure 4: This figure illustrates the procedure by which agents select actions during training.
  • Figure 5: The angular avoidance fitness score is tested for different margin and cap values. The lighter lines represent a high margin and cap value. The darker lines represent a low margin and cap value.
  • ...and 11 more figures