Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Watch Your Step: Learning Semantically-Guided Locomotion in Cluttered Environment

Denan Liang, Yuan Zhu, Ruimeng Liu, Thien-Minh Nguyen, Shenghai Yuan, Lihua Xie

TL;DR

SemLoco, a Reinforcement Learning (RL) framework designed to avoid obstacles precisely in densely cluttered environments, significantly reduces collisions and improves safety around sensitive objects, enabling reliable navigation in situations where traditional controllers would likely cause damage.

Abstract

Although legged robots demonstrate impressive mobility on rough terrain, using them safely in cluttered environments remains a challenge. A key issue is their inability to avoid stepping on low-lying objects, such as high-cost small devices or cables on flat ground. This limitation arises from a disconnection between high-level semantic understanding and low-level control, combined with errors in elevation maps during real-world operation. To address this, we introduce SemLoco, a Reinforcement Learning (RL) framework designed to avoid obstacles precisely in densely cluttered environments. SemLoco uses a two-stage RL approach that combines both soft and hard constraints and performs pixel-wise foothold safety inference, enabling more accurate foot placement. Additionally, SemLoco integrates a semantic map to assign traversability costs rather than relying solely on geometric data. SemLoco significantly reduces collisions and improves safety around sensitive objects, enabling reliable navigation in situations where traditional controllers would likely cause damage. Experimental results further demonstrate that SemLoco can be effectively applied to more complex, unstructured real-world environments.

Watch Your Step: Learning Semantically-Guided Locomotion in Cluttered Environment

TL;DR

SemLoco, a Reinforcement Learning (RL) framework designed to avoid obstacles precisely in densely cluttered environments, significantly reduces collisions and improves safety around sensitive objects, enabling reliable navigation in situations where traditional controllers would likely cause damage.

Abstract

Although legged robots demonstrate impressive mobility on rough terrain, using them safely in cluttered environments remains a challenge. A key issue is their inability to avoid stepping on low-lying objects, such as high-cost small devices or cables on flat ground. This limitation arises from a disconnection between high-level semantic understanding and low-level control, combined with errors in elevation maps during real-world operation. To address this, we introduce SemLoco, a Reinforcement Learning (RL) framework designed to avoid obstacles precisely in densely cluttered environments. SemLoco uses a two-stage RL approach that combines both soft and hard constraints and performs pixel-wise foothold safety inference, enabling more accurate foot placement. Additionally, SemLoco integrates a semantic map to assign traversability costs rather than relying solely on geometric data. SemLoco significantly reduces collisions and improves safety around sensitive objects, enabling reliable navigation in situations where traditional controllers would likely cause damage. Experimental results further demonstrate that SemLoco can be effectively applied to more complex, unstructured real-world environments.
Paper Structure (25 sections, 14 equations, 5 figures, 1 table)

This paper contains 25 sections, 14 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Reinforcement Learning
  5. Observation Space
  6. Action Space
  7. Network Architecture
  8. Semantic-Aware Adaptive Foothold Planning
  9. Reward Functions
  10. Primary Objective Rewards ($r_{\text{primary}}$)
  11. State and Dynamic Constraint Penalties ($r_{\text{penalty}}$)
  12. Learning Semantic-Aware Locomotion via Two-Stage RL
  13. Virtual Obstacle Stage
  14. Rigid Obstacle Stage
  15. Curriculum Learning for Generalization and Robustness
  16. ...and 10 more sections

Figures (5)

  • Figure 1: SemLoco Overview for semantic-aware prediction in cluttered environments. This figure demonstrates a quadruped robot performing obstacle avoidance in a real-world scene filled with small sensitive objects. Compared to the traditional pure elevation map, SemLoco integrates semantic map to low-level control, enabling the controller to perform pixel-wise foothold safety inference. This allows for precise gait planning in both simulation and real-world experiments, preventing damage to valuable items.
  • Figure 2: Framework of SemLoco. Sub-modules have different styles based on their functions. Among them, the red trapezoid represents the neural network, the blue rectangle represents unprocessed raw data, and the green rectangle represents processed ready-to-use data. (a) Training in the simulator: In stage 1, we use virtual obstacles (Highly yellow spheres). Although the robot walks on flat ground, it receives a virtual perception map containing height and semantic information to simulate scenarios with real obstacles. In stage 2, we use rigid obstacles. The robot receives corresponding real perception information, and fine-tunes the policy to improve task performance. (b) Deployment in the real world: The semantic algorithm uses Odin1 to process exteroception. Combined with proprioception, this information is fed into the trained policy to generate low-level control commands.
  • Figure 3: Virtual semantic elevation map. Traversability are color-coded (red and yellow, respectively), with obstacle elevations represented by the vertical positions of the yellow spheres.
  • Figure 4: Example of obstacle curricula.
  • Figure 5: Performance contrast among different policies in real world.