Crossing the Sim2Real Gap Between Simulation and Ground Testing to Space Deployment of Autonomous Free-flyer Control

TL;DR

The paper tackles the Sim2Real gap in space robotics by developing a PPO-based DRL controller trained in a high-fidelity zero-G simulator (NVIDIA Omniverse Isaac Lab) with curriculum learning to replace Astrobee's GNC. It validates the approach through simulation, ground testing at Granite Lab, and an on-orbit ISS demonstration, showing that curriculum-trained policies are robust to mass variations and can operate autonomously in microgravity. The results establish a complete sim-to-hardware pipeline for RL-based autonomous space robotics and demonstrate the feasibility of rapid on-orbit adaptation for ISAM missions. Together, these findings advance autonomous AI&T concepts and lay groundwork for future ISAM-enabled robotic space missions.

Abstract

Reinforcement learning (RL) offers transformative potential for robotic control in space. We present the first on-orbit demonstration of RL-based autonomous control of a free-flying robot, the NASA Astrobee, aboard the International Space Station (ISS). Using NVIDIA's Omniverse physics simulator and curriculum learning, we trained a deep neural network to replace Astrobee's standard attitude and translation control, enabling it to navigate in microgravity. Our results validate a novel training pipeline that bridges the simulation-to-reality (Sim2Real) gap, utilizing a GPU-accelerated, scientific-grade simulation environment for efficient Monte Carlo RL training. This successful deployment demonstrates the feasibility of training RL policies terrestrially and transferring them to space-based applications. This paves the way for future work in In-Space Servicing, Assembly, and Manufacturing (ISAM), enabling rapid on-orbit adaptation to dynamic mission requirements.

Figures (6)

• Figure 1: Astrobee Flight Control Reinforcement Learning with PPO
• Figure 2: MSE comparison of baseline controller and various RL policies for robustness to mass variation: (a) Baseline Astrobee Controller, (b) no curriculum RL Policy, (c) iss tested Mass Variation (+- 1.5kg) Curriculum, (d) more mass var (+- 9kg) Curriculum
• Figure 3: Ground testing of RL control on Astrobee hardware at NASA Ames Granite Lab. (a) Docked state with robot without a payload. (b) Undocked state of robot without a payload (c) Robot including the robotic arm payload attached.
• Figure 4: Dataset comparing undock with (solid lines) and without (dotted lines) the arm mass payload.
• Figure 5: Astrobee free-flyer under RL control in zero-G on-board the ISS.