Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DribbleBot: Dynamic Legged Manipulation in the Wild

Yandong Ji, Gabriel B. Margolis, Pulkit Agrawal

TL;DR

DribbleBot demonstrates dynamic, in-the-wild dribbling by a quadruped, trained in simulation with PPO and robust sim-to-real transfer techniques. The approach integrates ball-centric global velocity commands, body-mounted fisheye perception, and a recovery policy to handle falls, backed by domain randomization and a specialized ball-terrain drag model. Key contributions include a fully simulated-to-real dribbling pipeline on a small quadruped, a fine-tuned fisheye-based ball detector, and ablations that illuminate the importance of perception, drag modeling, and recovery in real-world robustness. The work establishes a practical baseline for in-the-wild dynamic mobile manipulation and motivates further integration of high-level play, environment awareness, and human-robot interaction in legged soccer tasks.

Abstract

DribbleBot (Dexterous Ball Manipulation with a Legged Robot) is a legged robotic system that can dribble a soccer ball under the same real-world conditions as humans (i.e., in-the-wild). We adopt the paradigm of training policies in simulation using reinforcement learning and transferring them into the real world. We overcome critical challenges of accounting for variable ball motion dynamics on different terrains and perceiving the ball using body-mounted cameras under the constraints of onboard computing. Our results provide evidence that current quadruped platforms are well-suited for studying dynamic whole-body control problems involving simultaneous locomotion and manipulation directly from sensory observations.

DribbleBot: Dynamic Legged Manipulation in the Wild

TL;DR

DribbleBot demonstrates dynamic, in-the-wild dribbling by a quadruped, trained in simulation with PPO and robust sim-to-real transfer techniques. The approach integrates ball-centric global velocity commands, body-mounted fisheye perception, and a recovery policy to handle falls, backed by domain randomization and a specialized ball-terrain drag model. Key contributions include a fully simulated-to-real dribbling pipeline on a small quadruped, a fine-tuned fisheye-based ball detector, and ablations that illuminate the importance of perception, drag modeling, and recovery in real-world robustness. The work establishes a practical baseline for in-the-wild dynamic mobile manipulation and motivates further integration of high-level play, environment awareness, and human-robot interaction in legged soccer tasks.

Abstract

DribbleBot (Dexterous Ball Manipulation with a Legged Robot) is a legged robotic system that can dribble a soccer ball under the same real-world conditions as humans (i.e., in-the-wild). We adopt the paradigm of training policies in simulation using reinforcement learning and transferring them into the real world. We overcome critical challenges of accounting for variable ball motion dynamics on different terrains and perceiving the ball using body-mounted cameras under the constraints of onboard computing. Our results provide evidence that current quadruped platforms are well-suited for studying dynamic whole-body control problems involving simultaneous locomotion and manipulation directly from sensory observations.
Paper Structure (29 sections, 4 figures, 5 tables)

This paper contains 29 sections, 4 figures, 5 tables.

Table of Contents

  1. INTRODUCTION
  2. MATERIALS
  3. METHOD
  4. Training the Dribbling Policy
  5. Environment Design
  6. Control Interface for Dribbling in the Wild
  7. Observation and Action Space
  8. Reward Model
  9. Policy Architecture and Optimization
  10. Measures to Mitigate the Sim-to-Real Gap
  11. Visual Perception in Fisheye Images
  12. Vision Noise Model
  13. Robot System Identification
  14. Ball-Terrain Interaction Model
  15. Fall Recovery Controller
  16. ...and 14 more sections

Figures (4)

  • Figure 1: Measures added for sim-to-real transfer in DribbleBot.
  • Figure 2: System architecture for DribbleBot.$\pi_d$ and $\pi_r$ are multilayer perceptrons trained using reinforcement learning in simulation. YOLOv7 is an object detection network wang2022yolov7 that we fine-tune on images from our domain using supervised learning.
  • Figure 3:
  • Figure 8: Overhead images of DribbleBot during real-world deployment.