Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots

Ashutosh Mishra, Shreya Santra, Elian Neppel, Edoardo M. Rossi Lombardi, Shamistan Karimov, Kentaro Uno, Kazuya Yoshida

TL;DR

This work tackles autonomous control of modular, reconfigurable lunar robots by introducing a fully decentralized reinforcement learning framework (Dec-RL) in which each module learns its own policy (wheel SAC, steering limb PPO, manipulation limb PPO) and policies are composed at runtime to support multiple morphologies. The approach achieves zero-shot generalization to unseen configurations and demonstrates robust coordination across synchronous, parallel, and sequential execution modes, validated through simulations and lunar-analog hardware tests. Key contributions include a scalable, morphology-agnostic policy architecture, enhanced sample efficiency over centralized baselines, and hardware validation showing energy and torque efficiency gains, as well as practical guidance for policy composition in dynamic morphologies. The results indicate a practical path toward autonomous, adaptable lunar infrastructures, with future work focusing on broader morphology coverage, higher sample efficiency, and lighter hardware to improve robustness in uncertain lunar environments.

Abstract

Modular reconfigurable robots suit task-specific space operations, but the combinatorial growth of morphologies hinders unified control. We propose a decentralized reinforcement learning (Dec-RL) scheme where each module learns its own policy: wheel modules use Soft Actor-Critic (SAC) for locomotion and 7-DoF limbs use Proximal Policy Optimization (PPO) for steering and manipulation, enabling zero-shot generalization to unseen configurations. In simulation, the steering policy achieved a mean absolute error of 3.63° between desired and induced angles; the manipulation policy plateaued at 84.6 % success on a target-offset criterion; and the wheel policy cut average motor torque by 95.4 % relative to baseline while maintaining 99.6 % success. Lunar-analogue field tests validated zero-shot integration for autonomous locomotion, steering, and preliminary alignment for reconfiguration. The system transitioned smoothly among synchronous, parallel, and sequential modes for Policy Execution, without idle states or control conflicts, indicating a scalable, reusable, and robust approach for modular lunar robots.

Multi-Modal Decentralized Reinforcement Learning for Modular Reconfigurable Lunar Robots

TL;DR

This work tackles autonomous control of modular, reconfigurable lunar robots by introducing a fully decentralized reinforcement learning framework (Dec-RL) in which each module learns its own policy (wheel SAC, steering limb PPO, manipulation limb PPO) and policies are composed at runtime to support multiple morphologies. The approach achieves zero-shot generalization to unseen configurations and demonstrates robust coordination across synchronous, parallel, and sequential execution modes, validated through simulations and lunar-analog hardware tests. Key contributions include a scalable, morphology-agnostic policy architecture, enhanced sample efficiency over centralized baselines, and hardware validation showing energy and torque efficiency gains, as well as practical guidance for policy composition in dynamic morphologies. The results indicate a practical path toward autonomous, adaptable lunar infrastructures, with future work focusing on broader morphology coverage, higher sample efficiency, and lighter hardware to improve robustness in uncertain lunar environments.

Abstract

Modular reconfigurable robots suit task-specific space operations, but the combinatorial growth of morphologies hinders unified control. We propose a decentralized reinforcement learning (Dec-RL) scheme where each module learns its own policy: wheel modules use Soft Actor-Critic (SAC) for locomotion and 7-DoF limbs use Proximal Policy Optimization (PPO) for steering and manipulation, enabling zero-shot generalization to unseen configurations. In simulation, the steering policy achieved a mean absolute error of 3.63° between desired and induced angles; the manipulation policy plateaued at 84.6 % success on a target-offset criterion; and the wheel policy cut average motor torque by 95.4 % relative to baseline while maintaining 99.6 % success. Lunar-analogue field tests validated zero-shot integration for autonomous locomotion, steering, and preliminary alignment for reconfiguration. The system transitioned smoothly among synchronous, parallel, and sequential modes for Policy Execution, without idle states or control conflicts, indicating a scalable, reusable, and robust approach for modular lunar robots.
Paper Structure (16 sections, 8 equations, 9 figures)

This paper contains 16 sections, 8 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Modular Robot System and Decentralized Policy Learning
  4. Wheel Module Policy
  5. Steering Limb Policy
  6. Manipulation limb Policy
  7. Morphology-Specific Constraints
  8. Policy Composition and Coordination
  9. Synchronous Parallel Execution
  10. Parallel Execution of Heterogeneous Policies
  11. Sequential Execution via Policy Gating
  12. Experimental Setup
  13. Results and Discussion
  14. Policy Performance during Learning
  15. Performance Evaluation on Hardware
  16. ...and 1 more sections

Figures (9)

  • Figure 1: MoonBot modules and policy: Wheel modules use SAC (Soft Actor-Critic) for locomotion and 7-DoF limbs use PPO (Proximal Policy Optimization) for manipulation and steering.
  • Figure 2: Simulation environment for modular robot components, used for individual policy learning and testing. Wheel locomotion and Dragon-mode steering were learned in MuJoCo, while manipulation and Vehicle-mode steering tasks were implemented in NVIDIA Isaac Lab. (a) Wheel dynamics simulation. (b) Manipulator grasping. (c) Dragon steering. (d) Vehicle limb-steering.
  • Figure 3: MoonBot reconfiguration in process: Phase 1: Dragon approaches with $\pi_{\text{wheel}}$ and $\pi_{\text{steer}}$, Phase 2: manipulator limb aligns with grasping fixture using $\pi_{\text{manip}}$, Phase 3: manipulator limb is detached using teleoperation to form Vehicle and Minimal modes.
  • Figure 4: Steering compliance of the 7-DoF limb Policy: induced vs. desired curvature.
  • Figure 5: Relative distance between end-effector and target across varying initial distances.
  • ...and 4 more figures