Training and Simulation of Quadrupedal Robot in Adaptive Stair Climbing for Indoor Firefighting: An End-to-End Reinforcement Learning Approach
Baixiao Huang, Baiyu Huang, Yu Hou
TL;DR
This work targets indoor firefighting primary search by enabling a quadrupedal robot to autonomously navigate and climb stairs using an end-to-end reinforcement learning framework. It introduces a two-stage training pipeline that transfers skills from pyramid-stair terrain to diverse indoor stair geometries, coupled with a centerline-based navigation approach and local height-map perception. The policy, trained with PPO in the NVIDIA Isaac Lab, uses a CNN+MLP architecture to produce joint position commands, guided by a comprehensive reward structure and curriculum. Results show that the two-stage training improves performance across straight, L-shaped, and spiral stairs, though higher stair levels and spirals remain challenging, highlighting both the method’s potential for real-world firefighting and areas for further Sim-to-Real validation.
Abstract
Quadruped robots are used for primary searches during the early stages of indoor fires. A typical primary search involves quickly and thoroughly looking for victims under hazardous conditions and monitoring flammable materials. However, situational awareness in complex indoor environments and rapid stair climbing across different staircases remain the main challenges for robot-assisted primary searches. In this project, we designed a two-stage end-to-end deep reinforcement learning (RL) approach to optimize both navigation and locomotion. In the first stage, the quadrupeds, Unitree Go2, were trained to climb stairs in Isaac Lab's pyramid-stair terrain. In the second stage, the quadrupeds were trained to climb various realistic indoor staircases in the Isaac Lab engine, with the learned policy transferred from the previous stage. These indoor staircases are straight, L-shaped, and spiral, to support climbing tasks in complex environments. This project explores how to balance navigation and locomotion and how end-to-end RL methods can enable quadrupeds to adapt to different stair shapes. Our main contributions are: (1) A two-stage end-to-end RL framework that transfers stair-climbing skills from abstract pyramid terrain to realistic indoor stair topologies. (2) A centerline-based navigation formulation that enables unified learning of navigation and locomotion without hierarchical planning. (3) Demonstration of policy generalization across diverse staircases using only local height-map perception. (4) An empirical analysis of success, efficiency, and failure modes under increasing stair difficulty.