Dribble Master: Learning Agile Humanoid Dribbling Through Legged Locomotion

TL;DR

The paper introduces Dribble Master, a two-stage curriculum reinforcement learning framework that enables agile humanoid dribbling through loco-manipulation. A virtual camera model and active-sensing rewards are used to train robust ball perception and control in simulation, with sim-to-real transfer to Booster T1 demonstrated. The approach shows high velocity-tracking accuracy, strong real-world performance, and robust generalization across hardware and terrains. Key contributions include the two-stage curriculum, active vision strategies, and comprehensive sim-to-real bridging techniques, offering a practical path toward versatile humanoid soccer robots.

Abstract

Humanoid soccer dribbling is a highly challenging task that demands dexterous ball manipulation while maintaining dynamic balance. Traditional rule-based methods often struggle to achieve accurate ball control due to their reliance on fixed walking patterns and limited adaptability to real-time ball dynamics. To address these challenges, we propose a two-stage curriculum learning framework that enables a humanoid robot to acquire dribbling skills without explicit dynamics or predefined trajectories. In the first stage, the robot learns basic locomotion skills; in the second stage, we fine-tune the policy for agile dribbling maneuvers. We further introduce a virtual camera model in simulation that simulates the field of view and perception constraints of the real robot, enabling realistic ball perception during training. We also design heuristic rewards to encourage active sensing, promoting a broader visual range for continuous ball perception. The policy is trained in simulation and successfully transferred to a physical humanoid robot. Experiment results demonstrate that our method enables effective ball manipulation, achieving flexible and visually appealing dribbling behaviors across multiple environments. This work highlights the potential of reinforcement learning in developing agile humanoid soccer robots. Additional details and videos are available at https://zhuoheng0910.github.io/dribble-master/.

Figures (7)

• Figure 1: Dribble Master: Humanoid robot learning to dribble under various tasks.(a): The robot receives ball velocity commands via a joystick and successfully navigates an array of three obstacles, ultimately dribbling the ball to the goal with high agility and precision. (b)(c): The robot exhibits agile directional-change capabilities. (b) combining straight-line movement with left-front and right-front turns; (c) combining straight-line movement with sharp left and right turns. (d)(e)(f): Detailed motion sequence showing the robot executing a rightward direction change by making contact with the ball.
• Figure 2: System Architecture of Dribble Master. In the phase of training in simulation, we use a two-stage learning approach. During the 1st stage, the locomotion rewards are given higher weights, with the ball far away from the robot. This aims to train the robot to run to the ball stably and rapidly. During the 2nd stage, the balls are near the robot and the dribbling rewards have non-zero high weights, which enables the robot to learn to manipulate the ball for precise velocity tracking. In the phase of real-world deployment, we transfer the trained actor policy to a physical Booster T1 humanoid robot. The ball position is obtained via the YOLOv8 module for ball detection of raw images. The target ball velocity is commanded by a joystick maneuvered by humans.
• Figure 3: Active sensing rewards encourage the robot to search for the ball. When the ball is not in the view((a) and (b)), the robot has a smaller reward than (c) and (d).
• Figure 4: Mean ball trajectories (solid blue and orange lines) for four dribbling maneuvers. The blue and orange shaded regions represent the variation across all individual trajectories for the 45-degree turn and the 90-degree turn, respectively.
• Figure 5: Ablation study tasks setup.(a)(b) The three target positions and the region definition of success and failure in the Dribbling to Target task. (c) The region setup in the Obstacle Avoidance task.