Music-Driven Legged Robots: Synchronized Walking to Rhythmic Beats

TL;DR

The paper addresses the challenge of making legged robots walk in sync with musical beats by introducing a hierarchical controller that decouples frequency tracking from phase alignment. A low-level phase tracker with distributed oscillators learns to follow nominal frequencies, while a high-level phase modulator aligns the robot’s internal phase with music, supported by a ground reaction force estimator to operate without precise force sensing. Key contributions include a two-stage training pipeline, a robust rhythm-consistency reward, and real-time sim-to-real demonstrations on a Go1 platform, validating beat-aligned gait across frequencies and musical tracks. This work advances real-time rhythmic locomotion for quadrupeds, enabling applications such as dancing and music-driven exploration with improved controllability and robustness in physical robots.

Abstract

We address the challenge of effectively controlling the locomotion of legged robots by incorporating precise frequency and phase characteristics, which is often ignored in locomotion policies that do not account for the periodic nature of walking. We propose a hierarchical architecture that integrates a low-level phase tracker, oscillators, and a high-level phase modulator. This controller allows quadruped robots to walk in a natural manner that is synchronized with external musical rhythms. Our method generates diverse gaits across different frequencies and achieves real-time synchronization with music in the physical world. This research establishes a foundational framework for enabling real-time execution of accurate rhythmic motions in legged robots. Video is available at website: https://music-walker.github.io/.

Figures (8)

• Figure 1: Visualization of music features sampled at 100 Hz, including the envelope (music intensity), smoothed beats, and interpolated phase information, based on a 3-second music clip with a tempo of 120 BPM.
• Figure 2: Proposed controller, features a hierarchical structure with three main modules: the high-level Phase Synchronization Modulator, the low-level Phase Tracking Policy, and the Distributed Phase Oscillators. In the figure, the top-left and top-right corners respectively depict the music phase ring and the robot's oscillator phase ring, both with a range of $0$ to $2\pi$. The green dot on the music phase ring represents the current music frame, and its features (such as beat and phase observations) are sent to the high-level network. The four points on the robot's phase ring correspond to the phases of the oscillators bound to each of the four legs. Oscillator Num.1 (associated with the right front leg) is tasked with synchronizing with the musical beat.
• Figure 3: Robot’s oscillator phase loop within range [0, 2$\pi$].
• Figure 4: Frequency tracking Sim2Real Experiments conducted on Go1 robot with Low-Level Networks and Oscillators. Our controller demonstrates diverse gait skills. The robot experiment videos are recorded at 60 fps. Here, we have extracted five images from videos of 4.0 Hz and 1.5 Hz frequencies commands, with each image sampled every 10 frames.
• Figure 5: Ground reaction force (GRF) data collected over 2,500 time steps (5 seconds). (a) Shows normalized GRF values at a command frequency of 2.0 Hz, highlighting clear periodic patterns. (b) Displays frequency deviation data across six different command frequencies.