Learning Force Control for Legged Manipulation

TL;DR

This work introduces a learned whole-body force-control framework for a legged robot with an attached arm, enabling direct end-effector force commands without force sensors. A single policy is trained in simulation to switch between force and position tasks, using a soft contact model and a composite reward, and is deployed on a real quadruped to achieve gravity compensation, impedance-like behavior, and compliant manipulation. Results show force-tracking accuracy around 5–10 N, a substantial expansion of the manipulation workspace (59%), and practical teleoperation benefits via impedance control and kinesthetic demonstrations. The approach offers a data-efficient path toward versatile, safe legged loco-manipulation with minimal sensing, highlighting a path for imitation-based autonomous learning in the future.

Abstract

Controlling contact forces during interactions is critical for locomotion and manipulation tasks. While sim-to-real reinforcement learning (RL) has succeeded in many contact-rich problems, current RL methods achieve forceful interactions implicitly without explicitly regulating forces. We propose a method for training RL policies for direct force control without requiring access to force sensing. We showcase our method on a whole-body control platform of a quadruped robot with an arm. Such force control enables us to perform gravity compensation and impedance control, unlocking compliant whole-body manipulation. The learned whole-body controller with variable compliance makes it intuitive for humans to teleoperate the robot by only commanding the manipulator, and the robot's body adjusts automatically to achieve the desired position and force. Consequently, a human teleoperator can easily demonstrate a wide variety of loco-manipulation tasks. To the best of our knowledge, we provide the first deployment of learned whole-body force control in legged manipulators, paving the way for more versatile and adaptable legged robots.

Figures (4)

• Figure 1: System architecture. In addition to the position command $\textbf{p}_{ee}^{cmd}$, we consider training a policy to track a commanded end effector force $\textbf{F}^{cmd}$ while walking following a base velocity command $\textbf{v}_{b}^{cmd}$. A human operator defines the commands using a joystick or hand-tracking headset park2024avp. To learn force tracking, we train the policy in a potential field where the gripper is pulled towards a randomized setpoint $x_s$ (right).
• Figure 2: Simulated force control evaluation. Average force tracking and estimation errors for target forces applied at 1000 setpoints sampled across the full training workspace. The bars represent the standard deviation. The accompanying illustration (right) shows how the robot applies force at the gripper along the z-axis.
• Figure 3: Real-world deployment. We deploy our policy in a variety of real-world scenarios: (A) water pouring in a kitchenette; (B) tracking a long end effector trajectory in position control mode with performance verified by motion capture; (C) outdoor whole-body reaching; (D, E) quantitative measurement of the force application control by pulling down on a dynamometer; (F) lifting a 4kg box while maintaining compliance in the gripper, showcasing gravity compensation. In subfigure (D), each line represents the mean across five force setpoints, with the shaded area indicating the standard deviation.
• Figure 4: Expanded workspace. (A-B) The whole-body policy in position control mode (red) enlarges the fixed arm workspace (green) by 59%. (C) The position error distribution shows the tracking accuracy while standing in place, evaluated across 1000 end effector positions sampled from the training distribution.