Latent Space Reinforcement Learning for Multi-Robot Exploration

TL;DR

The paper tackles scalable autonomous exploration by multiple robots in unknown environments. It introduces a latent-space reinforcement learning framework that uses a convolutional autoencoder to compress high-resolution occupancy grids into a 2048-dimensional latent state, enabling tractable multi-agent DRL. A novel Perlin-noise based map generator creates diverse, large-scale training environments, while a hierarchical RL architecture with a trust-based weighted consensus enables robust decentralized coordination despite noisy communication. Experimental results show the system scales with the number of agents, generalizes to unfamiliar topologies, and remains resilient to communication noise, highlighting practical potential for scalable, robust multi-robot exploration in complex terrains.

Abstract

Autonomous mapping of unknown environments is a critical challenge, particularly in scenarios where time is limited. Multi-agent systems can enhance efficiency through collaboration, but the scalability of motion-planning algorithms remains a key limitation. Reinforcement learning has been explored as a solution, but existing approaches are constrained by the limited input size required for effective learning, restricting their applicability to discrete environments. This work addresses that limitation by leveraging autoencoders to perform dimensionality reduction, compressing high-fidelity occupancy maps into latent state vectors while preserving essential spatial information. Additionally, we introduce a novel procedural generation algorithm based on Perlin noise, designed to generate topologically complex training environments that simulate asteroid fields, caves and forests. These environments are used for training the autoencoder and the navigation algorithm using a hierarchical deep reinforcement learning framework for decentralized coordination. We introduce a weighted consensus mechanism that modulates reliance on shared data via a tuneable trust parameter, ensuring robustness to accumulation of errors. Experimental results demonstrate that the proposed system scales effectively with number of agents and generalizes well to unfamiliar, structurally distinct environments and is resilient in communication-constrained settings.

Figures (17)

• Figure 1: An example scenario. (i) To the left: 10 robots exploring the ground map. One agent is highlighted with a circle to visualize the sensing radius. (ii) To the right: From top-left, the highlighted agent's fused obstacle map, fused exploration map, self obstacle map, self exploration map. The final two images are relevant to the autoencoder as discussed in Section 4.
• Figure 2: An example usage of the map generation algorithm. The labels are as used in Algorithm 1.
• Figure 3: Examples of generated maps with varying difficulty levels.
• Figure 4: A block diagram of a typical autoencoder.
• Figure 5: Generalization test performed on the autoencoder. The top row displays the original grayscale images fed into the encoder. The bottom row displays the reconstructed images after the encoded latent vectors are fed into the decoder.