Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sensor to Pixels: Decentralized Swarm Gathering via Image-Based Reinforcement Learning

Yigal Koifman, Eran Iceland, Erez Koifman, Ariel Barel, Alfred M. Bruckstein

TL;DR

This work tackles the problem of converging a swarm with bearing-only, limited-range sensing while preserving connectivity in a decentralized setting. It introduces the Sensor to Pixels framework, which encodes local observations as structured images and processes them with a CNN within a CTDE MARL paradigm, using a continuous action space and a two-component reward that balances local cohesion and global convergence. Empirically, the method achieves fast convergence and strong connectivity, outperforming the analytical solution in many scenarios and offering competitive or superior performance to VariAntNet, especially under challenging conditions. The results demonstrate the practicality of image-based perception for scalable, decentralized swarm control with potential applications in time-critical missions requiring robust coordination.

Abstract

This study highlights the potential of image-based reinforcement learning methods for addressing swarm-related tasks. In multi-agent reinforcement learning, effective policy learning depends on how agents sense, interpret, and process inputs. Traditional approaches often rely on handcrafted feature extraction or raw vector-based representations, which limit the scalability and efficiency of learned policies concerning input order and size. In this work we propose an image-based reinforcement learning method for decentralized control of a multi-agent system, where observations are encoded as structured visual inputs that can be processed by Neural Networks, extracting its spatial features and producing novel decentralized motion control rules. We evaluate our approach on a multi-agent convergence task of agents with limited-range and bearing-only sensing that aim to keep the swarm cohesive during the aggregation. The algorithm's performance is evaluated against two benchmarks: an analytical solution proposed by Bellaiche and Bruckstein, which ensures convergence but progresses slowly, and VariAntNet, a neural network-based framework that converges much faster but shows medium success rates in hard constellations. Our method achieves high convergence, with a pace nearly matching that of VariAntNet. In some scenarios, it serves as the only practical alternative.

Sensor to Pixels: Decentralized Swarm Gathering via Image-Based Reinforcement Learning

TL;DR

This work tackles the problem of converging a swarm with bearing-only, limited-range sensing while preserving connectivity in a decentralized setting. It introduces the Sensor to Pixels framework, which encodes local observations as structured images and processes them with a CNN within a CTDE MARL paradigm, using a continuous action space and a two-component reward that balances local cohesion and global convergence. Empirically, the method achieves fast convergence and strong connectivity, outperforming the analytical solution in many scenarios and offering competitive or superior performance to VariAntNet, especially under challenging conditions. The results demonstrate the practicality of image-based perception for scalable, decentralized swarm control with potential applications in time-critical missions requiring robust coordination.

Abstract

This study highlights the potential of image-based reinforcement learning methods for addressing swarm-related tasks. In multi-agent reinforcement learning, effective policy learning depends on how agents sense, interpret, and process inputs. Traditional approaches often rely on handcrafted feature extraction or raw vector-based representations, which limit the scalability and efficiency of learned policies concerning input order and size. In this work we propose an image-based reinforcement learning method for decentralized control of a multi-agent system, where observations are encoded as structured visual inputs that can be processed by Neural Networks, extracting its spatial features and producing novel decentralized motion control rules. We evaluate our approach on a multi-agent convergence task of agents with limited-range and bearing-only sensing that aim to keep the swarm cohesive during the aggregation. The algorithm's performance is evaluated against two benchmarks: an analytical solution proposed by Bellaiche and Bruckstein, which ensures convergence but progresses slowly, and VariAntNet, a neural network-based framework that converges much faster but shows medium success rates in hard constellations. Our method achieves high convergence, with a pace nearly matching that of VariAntNet. In some scenarios, it serves as the only practical alternative.
Paper Structure (18 sections, 3 equations, 5 figures, 3 tables)

This paper contains 18 sections, 3 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setting
  4. Notation and Definitions
  5. Methodology
  6. Observation Space
  7. Action Space
  8. Network Architecture
  9. Feature Extraction
  10. Actor-Critic Network
  11. Reward Function
  12. Results
  13. Training Parameters
  14. Training Evaluation
  15. Execution Results
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Convergence of 20 agents. Left: local observations of a typical agent, where white pixels indicate detected neighbors within a limited sensing range. Right: global view of the swarm’s convergence trace.
  • Figure 2: Sensor to Pixels framework.
  • Figure 3: Preprocessing from local observation to pixel-grid representation.
  • Figure 4: Actor Critic NN Architecture.
  • Figure 5: Training process evaluation of 10 and 20 agents after a convergence to a stable policy. From each training session, a single checkpoint model was selected for further evaluation.