Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision

Yutong Hu, Kehan Wen, Fisher Yu

TL;DR

DexDribbler tackles the challenge of learning dynamic, dexterous ball manipulation with legged robots by injecting a body-level dynamic supervision signal into reinforcement learning. The framework couples a context estimator with a policy, uses domain randomization, and employs a neural-aided Kalman filter to enable robust sim-to-real transfer, achieving faster convergence and the ability to perform sharp cuts and turns on flat surfaces. Real-world deployment demonstrates zero-shot transfer across terrains with a perception fusion pipeline, outperforming prior soccer-based learning approaches in both simulation and hardware. This work broadens the scope of legged manipulation by integrating high-level dynamical guidance into learning, with potential applicability to a wide range of multi-task, contact-rich robotic tasks.

Abstract

Learning dexterous locomotion policy for legged robots is becoming increasingly popular due to its ability to handle diverse terrains and resemble intelligent behaviors. However, joint manipulation of moving objects and locomotion with legs, such as playing soccer, receive scant attention in the learning community, although it is natural for humans and smart animals. A key challenge to solve this multitask problem is to infer the objectives of locomotion from the states and targets of the manipulated objects. The implicit relation between the object states and robot locomotion can be hard to capture directly from the training experience. We propose adding a feedback control block to compute the necessary body-level movement accurately and using the outputs as dynamic joint-level locomotion supervision explicitly. We further utilize an improved ball dynamic model, an extended context-aided estimator, and a comprehensive ball observer to facilitate transferring policy learned in simulation to the real world. We observe that our learning scheme can not only make the policy network converge faster but also enable soccer robots to perform sophisticated maneuvers like sharp cuts and turns on flat surfaces, a capability that was lacking in previous methods. Video and code are available at https://github.com/SysCV/soccer-player

DexDribbler: Learning Dexterous Soccer Manipulation via Dynamic Supervision

TL;DR

DexDribbler tackles the challenge of learning dynamic, dexterous ball manipulation with legged robots by injecting a body-level dynamic supervision signal into reinforcement learning. The framework couples a context estimator with a policy, uses domain randomization, and employs a neural-aided Kalman filter to enable robust sim-to-real transfer, achieving faster convergence and the ability to perform sharp cuts and turns on flat surfaces. Real-world deployment demonstrates zero-shot transfer across terrains with a perception fusion pipeline, outperforming prior soccer-based learning approaches in both simulation and hardware. This work broadens the scope of legged manipulation by integrating high-level dynamical guidance into learning, with potential applicability to a wide range of multi-task, contact-rich robotic tasks.

Abstract

Learning dexterous locomotion policy for legged robots is becoming increasingly popular due to its ability to handle diverse terrains and resemble intelligent behaviors. However, joint manipulation of moving objects and locomotion with legs, such as playing soccer, receive scant attention in the learning community, although it is natural for humans and smart animals. A key challenge to solve this multitask problem is to infer the objectives of locomotion from the states and targets of the manipulated objects. The implicit relation between the object states and robot locomotion can be hard to capture directly from the training experience. We propose adding a feedback control block to compute the necessary body-level movement accurately and using the outputs as dynamic joint-level locomotion supervision explicitly. We further utilize an improved ball dynamic model, an extended context-aided estimator, and a comprehensive ball observer to facilitate transferring policy learned in simulation to the real world. We observe that our learning scheme can not only make the policy network converge faster but also enable soccer robots to perform sophisticated maneuvers like sharp cuts and turns on flat surfaces, a capability that was lacking in previous methods. Video and code are available at https://github.com/SysCV/soccer-player
Paper Structure (31 sections, 8 equations, 7 figures, 2 tables)

This paper contains 31 sections, 8 equations, 7 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Dynamic Object Manipulation
  4. Legged Locomotion
  5. Legs as Manipulators: Robot Soccer
  6. Overview
  7. Training Phase in Simulator
  8. Environment Design
  9. State Definition
  10. Ball-Terrain Interaction Model
  11. Context Definition
  12. Reward Design
  13. Feedback Controller in the Learning Loop
  14. Neural Network Architecture and Optimization:
  15. Deployment Phase in Real World
  16. ...and 16 more sections

Figures (7)

  • Figure 1: Demonstration of a DexDribbler. Guided by a feedback controller, the robot learns pinpoint coordination between body movement and feet motion in simulator. This enables it to execute "deliberate overshooting" --- a critical technique for performing sharp cuts and turns while dribbling on flat and smooth surfaces in real world.
  • Figure 2: Training pipeline during learning phase in simulator. Ground truth states can be obtained from simulator, and are used both for estimator network supervision, and for body speed feed-back computation.
  • Figure 3: Deployment pipeline in real world. The ball position vector is calculated by a kalman filter combining: (a) Constant Velocity Model (b) Projection-Intersection Model (c) Viewing Angle Model
  • Figure 4: Reward curve during training phase. (a) Task related reward term (b) Summation of all reward terms
  • Figure 5: Absolute ball velocity tracking error on different terrain.
  • ...and 2 more figures