Toward Control of Wheeled Humanoid Robots with Unknown Payloads: Equilibrium Point Estimation via Real-to-Sim Adaptation

TL;DR

The paper tackles the problem of controlling wheeled humanoid robots when lifting unknown payloads by explicitly predicting the new equilibrium point and integrating it into a model-based controller. It introduces a real-to-sim adaptation framework that builds a high-fidelity nonlinear dynamics model $g_{\boldsymbol{\zeta}}$ optimized by PSO to minimize the reality gap, and trains a time-series data-driven estimator (e.g., LSTM) to predict the new equilibrium point $\theta_{lin}$ from proprioceptive history $\mathbf{X}=[x_{t:T},\theta_{t:T}]^\top$ for online deployment. The estimator feeds a model-based controller (LQR) to improve balancing and tracking under unknown payloads, with experiments on a physical WIP and RaiSim demonstrating reduced sim-to-real discrepancy and enhanced stability. The approach shows that coupling a refined nonlinear model with data-driven equilibrium estimation can safely and efficiently adapt control to unknown dynamics without extra sensors, advancing practical deployment of wheeled humanoids in uncertain environments.

Abstract

Model-based controllers using a linearized model around the system's equilibrium point is a common approach in the control of a wheeled humanoid due to their less computational load and ease of stability analysis. However, controlling a wheeled humanoid robot while it lifts an unknown object presents significant challenges, primarily due to the lack of knowledge in object dynamics. This paper presents a framework designed for predicting the new equilibrium point explicitly to control a wheeled-legged robot with unknown dynamics. We estimated the total mass and center of mass of the system from its response to initially unknown dynamics, then calculated the new equilibrium point accordingly. To avoid using additional sensors (e.g., force torque sensor) and reduce the effort of obtaining expensive real data, a data-driven approach is utilized with a novel real-to-sim adaptation. A more accurate nonlinear dynamics model, offering a closer representation of real-world physics, is injected into a rigid-body simulation for real-to-sim adaptation. The nonlinear dynamics model parameters were optimized using Particle Swarm Optimization. The efficacy of this framework was validated on a physical wheeled inverted pendulum, a simplified model of a wheeled-legged robot. The experimental results indicate that employing a more precise analytical model with optimized parameters significantly reduces the gap between simulation and reality, thus improving the efficiency of a model-based controller in controlling a wheeled robot with unknown dynamics

Figures (8)

• Figure 1: Conceptual overview of the proposed method. The data-driven regression model learns to identify the new equilibrium point for a wheeled inverted pendulum from a high-fidelity simulation closely mirroring the real world. A physical version of this pendulum tests the framework's viability, suggesting its applicability in controlling a wheeled humanoid robot lifting an unknown object.
• Figure 2: Real-to-Sim Adaptation via a High Fidelity Simulation. (a) High-fidelity simulation is achieved by minimizing parametric modeling error, $\Delta \mathbf{\bm{x_p}}$, via updates to $\mathbf{\bm{\zeta}}$. The state trajectories of the system in two different domains are obtained from a physical system and a simulation, respectively, offline. The global optimization algorithm (e.g., PSO) is utilized to identify the parameters $\mathbf{\bm{\zeta}}$ to deduce the reality gap. (b) A data-driven model (e.g., LSTM) is trained to predict the new equilibrium point via supervised learning with the data obtained from a high-fidelity simulation. (c) The trained data-driven model estimates the new equilibrium point online in deployment.
• Figure 3: Visualizing the Nonlinear Dynamic Function $g_{\mathbf{\bm{\zeta}}}$. The selected parameter $\mathbf{\bm{\zeta}}$ is [0.15, 0.12, 0.2, 0.02, 0.01, 0.7]. The model exhibits non-responsiveness near zero speed due to the dead zone effect, with a behavior that is more nonlinear compared to the standard viscous and Coulomb friction model (Baseline). Additionally, a slight phase shift is observed, indicative of a delay.
• Figure 4: Sample Result of Data Trajectory Comparison Between a Simulation and the Real World. The last graph presents a comparison of data trajectories between simulation and reality for all methods. Our approach effectively narrows the reality gap and aligns the trajectory shape with the actual one. The residual error is attributed from coupled-dynamics, non-parametric nonlinear dynamics, and class-specific data variance. All data samples are collected from the WIP system using LQR control.
• Figure 5: A Physical Wheeled Inverted Pendulum Testbed. (a) Balancing (top) and tracking a sinusoidal reference task (bottom) were conducted in non-ideal and unknown dynamic situation. (b) A customized wheeled Inverted pendulum is made up of a motor, wheel, IMU, translation rail, and weight. Two additional weights (0.8kg, 1.6kg) can be attached and detached to a pole link to adjust the total mass and the position of the CoM.