Discrete-Time Hybrid Automata Learning: Legged Locomotion Meets Skateboarding

TL;DR

Discrete-Time Hybrid Automata Learning (DHAL) tackles mode-switching in hybrid dynamical systems for legged locomotion by learning a discrete mode selector and mode-specific dynamics online, without trajectory segmentation or event-function labeling. The framework combines a DHA with a $\\beta$-VAE dynamics encoder, a Beta-distribution policy, and a Multi-Critic PPO-based RL loop to handle contact-rich, underactuated skateboarding on a quadruped robot, achieving sim-to-real transfer. Key contributions include a unsupervised mode identification mechanism, a discrete-time formulation that unifies switching and dynamics, and empirical validation showing robust mode-aware control and intuitive mode decomposition consistent with physical intuition. The work advances practical hybrid-control learning for complex robotics tasks, enabling stable, adaptive locomotion across varied terrains and disturbances.

Abstract

Hybrid dynamical systems, which include continuous flow and discrete mode switching, can model robotics tasks like legged robot locomotion. Model-based methods usually depend on predefined gaits, while model-free approaches lack explicit mode-switching knowledge. Current methods identify discrete modes via segmentation before regressing continuous flow, but learning high-dimensional complex rigid body dynamics without trajectory labels or segmentation is a challenging open problem. This paper introduces Discrete-time Hybrid Automata Learning (DHAL), a framework to identify and execute mode-switching without trajectory segmentation or event function learning. Besides, we embedded it in reinforcement learning pipeline and incorporates a beta policy distribution and a multi-critic architecture to model contact-guided motions, exemplified by a challenging quadrupedal robot skateboard task. We validate our method through sufficient real-world tests, demonstrating robust performance and mode identification consistent with human intuition in hybrid dynamical systems.

Figures (13)

• Figure 1: Demonstration of DHAL performance across various indoor and outdoor terrains, including slopes, carpets, sidewalks, step, and scenarios with additional payloads or disturbance. The controller enables the robot to perform smooth and natural skateboarding motions, with reliable mode identification and transitions under disturbances.
• Figure 2: The potholes on the downhill slope caused the robot dog's right front leg to get stuck, preventing it from smoothly getting onto the board. In fact, both of its hind legs even lost contact with the board. Nevertheless, the policy was still able to guide the robot dog to jump back onto the board and complete the recovery behavior.
• Figure 3: Discrete-time Hybrid Dynamics Learning (DHAL) Framework: (a) During training, the network learns to select the mode and activate the corresponding dynamics module (yellow-highlighted) to predict transition dynamics and contact. Here, $P_i$ represents the probability of the robot being in mode $i$ at time $t$. (b) The temporal features extracted by the encoder are combined with the current state and last action into the actor. The actor update $\alpha,\beta$, which define the probability density function of the Beta distribution, and then samples joint actions from the Beta distribution. (c) In a real-world deployment, we use different LED colors to indicate the active modes, showcasing smooth transitions and mode-specific behaviors.
• Figure 4: Switching of a hybrid system with inelastic collision. When the leg makes contact with the ground, the linear velocity will abruptly drop to zero.
• Figure 5: Multi-Critic Skateboard Task