Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning-based legged locomotion; state of the art and future perspectives

Sehoon Ha, Joonho Lee, Michiel van de Panne, Zhaoming Xie, Wenhao Yu, Majid Khadiv

TL;DR

A brief history of the field is given, recent efforts in learning locomotion skills for quadrupeds are summarized, and an understanding of the key issues involved is provided to provide researchers new to the area with an understanding of the key issues involved.

Abstract

Legged locomotion holds the premise of universal mobility, a critical capability for many real-world robotic applications. Both model-based and learning-based approaches have advanced the field of legged locomotion in the past three decades. In recent years, however, a number of factors have dramatically accelerated progress in learning-based methods, including the rise of deep learning, rapid progress in simulating robotic systems, and the availability of high-performance and affordable hardware. This article aims to give a brief history of the field, to summarize recent efforts in learning locomotion skills for quadrupeds, and to provide researchers new to the area with an understanding of the key issues involved. With the recent proliferation of humanoid robots, we further outline the rapid rise of analogous methods for bipedal locomotion. We conclude with a discussion of open problems as well as related societal impact.

Learning-based legged locomotion; state of the art and future perspectives

TL;DR

A brief history of the field is given, recent efforts in learning locomotion skills for quadrupeds are summarized, and an understanding of the key issues involved is provided to provide researchers new to the area with an understanding of the key issues involved.

Abstract

Legged locomotion holds the premise of universal mobility, a critical capability for many real-world robotic applications. Both model-based and learning-based approaches have advanced the field of legged locomotion in the past three decades. In recent years, however, a number of factors have dramatically accelerated progress in learning-based methods, including the rise of deep learning, rapid progress in simulating robotic systems, and the availability of high-performance and affordable hardware. This article aims to give a brief history of the field, to summarize recent efforts in learning locomotion skills for quadrupeds, and to provide researchers new to the area with an understanding of the key issues involved. With the recent proliferation of humanoid robots, we further outline the rapid rise of analogous methods for bipedal locomotion. We conclude with a discussion of open problems as well as related societal impact.
Paper Structure (53 sections, 1 equation, 6 figures, 2 tables)

This paper contains 53 sections, 1 equation, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Hardware
  3. Simulators
  4. Control and Learning Algorithms
  5. Related Survey Papers
  6. Theoretical background
  7. Markov Decision Process and Reinforcement Learning
  8. Deep Reinforcement Learning
  9. Behavior Cloning and Imitation Learning
  10. Components of MDP for Locomotion
  11. Dynamics
  12. Simulators
  13. Real robot
  14. Observation
  15. Proprioception
  16. ...and 38 more sections

Figures (6)

  • Figure 1: Examples of notable learned behaviors in the real world hoeller2023anymalxu2024dexterouszhuang2023robotmiki2022learningli2023learningunitree2024b2.
  • Figure 2: Evolution of quadruped hardware over time
  • Figure 3: Taxonomy of reinforcement learning algorithms. Popular algorithms are highlighted in color. On-policy algorithms (red), such as TRPO or PPO, have been the most frequent choice for legged locomotion. Other off-policy, gradient-free, or imitation learning algorithms (orange) are also selected for sample efficiency or to reproduce styles from example data. For further details, please refer to the background section.
  • Figure 4: A rough estimation of the usage of various simulators for locomotion learning. The numbers are collected using a Google Scholar search with the following keywords "simulator name" "legged" "robot" "hardware" and "learning". For the year 2024, as the numbers are for the first 9 months, we multiplied them by 4/3.
  • Figure 5: Illustration of popular learning frameworks: (a) basic learning, (b) curriculum learning, (c) hierarchical learning, and (d) privileged learning.
  • ...and 1 more figures