Learning Quiet Walking for a Small Home Robot

TL;DR

This work addresses the problem of loud footsteps in small home quadruped robots by proposing a sim-to-real reinforcement learning framework to minimize foot contact velocity $v_f$, a key proxy for footstep sound. The method combines adaptive PD gains, foot contact sensing, and curriculum learning, trained in the Isaac Gym simulator, with domain randomization to enhance transfer to hardware. Compared to a RL baseline and Sony controllers, the approach yields the quietest locomotion in the human-audible range ($20$ Hz to $20$ kHz) while revealing a trade-off with robustness to unknown terrains. The findings highlight practical pathways to quieter, more user-friendly home robots and suggest future work on perception-informed policy selection across varied surfaces.

Abstract

As home robotics gains traction, robots are increasingly integrated into households, offering companionship and assistance. Quadruped robots, particularly those resembling dogs, have emerged as popular alternatives for traditional pets. However, user feedback highlights concerns about the noise these robots generate during walking at home, particularly the loud footstep sound. To address this issue, we propose a sim-to-real based reinforcement learning (RL) approach to minimize the foot contact velocity highly related to the footstep sound. Our framework incorporates three key elements: learning varying PD gains to actively dampen and stiffen each joint, utilizing foot contact sensors, and employing curriculum learning to gradually enforce penalties on foot contact velocity. Experiments demonstrate that our learned policy achieves superior quietness compared to a RL baseline and the carefully handcrafted Sony commercial controllers. Furthermore, the trade-off between robustness and quietness is shown. This research contributes to developing quieter and more user-friendly robotic companions in home environments.

Figures (5)

• Figure 1: For the aibo small home robot pictured above, we design a sim-to-real based rl approach to minimize the foot contact velocity in the physics simulator, which highly correlates with the footstep sound in the real world to achieve quiet walking. Project webpage: https://sony.github.io/QuietWalk/
• Figure 2: System overview of the rl training framework for learning quiet locomotion with aibo. The agent, aibo, uses its IMU, joint encoders, and switch contact sensors to compute observations. As an action for 12 joints, the policy outputs target joint position and the joint PD gain scale, which enables the tracking of the target joints with high PD gain and the dampening of the joints with low PD gain. On the right is the Isaac Gym Makoviychuk2021-th simulation environment, which we leverage for parallel training on GPUs of multiple agents. The reward scales are divided into two phases, where aibo first learns just to walk, and then in the second phase, it adapts its walk to be quieter.
• Figure 3: Comparison of the average sound magnitude between $20$ Hz and $20$ kHz for the measured base velocity of aibo. The baseline and propose are RL policies without and with our three key factors for each. Sony normal and Sony quiet are the commercial controllers provided by Sony.
• Figure 4: Trade-off between robustness and quietness are shown. The experiment evaluates robustness by having aibo climb a slope. Sound magnitude is calculated using the same method as in Fig. \ref{['fig:sound_eval_vel']}. The controllers such as rl baseline, ablation test, SONY commercial controllers, and rl proposed are shown.
• Figure 5: PD gain scale of $\sigma(x)$ in the Eq. \ref{['eq:pd_gain_scale']} at the fore right leg during locomotion. Right foreleg is (A): in the air to move forward, (B): approaching contact with the ground, (C): in contact to support the ensuing movement of the other legs. The plot at the bottom shows the edge of the foot velocity. The blue area in the plot shows when the right foreleg is in contact with the ground, based on the foot switch contact sensor.