Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sample-Based Hybrid Mode Control: Asymptotically Optimal Switching of Algorithmic and Non-Differentiable Control Modes

Yilang Liu, Haoxiang You, Ian Abraham

TL;DR

The paper tackles the problem of dynamically switching between discrete, potentially non-differentiable and algorithmic control modes in robotics by formulating an indefinite hybrid mode switching problem and introducing a discrete-time, sample-based optimization approach. It redefines the problem through a discrete-time horizon with decisions over the mode, start time, and duration $(m,\tau,\lambda)$, and then develops an iterative, sample-based method that draws uniform candidates from the single-mode transition set $\Omega$ to achieve asymptotic convergence guarantees. Key contributions include a discrete-time definite formulation, an iterative sequencing scheme with local optimality properties, and a scalable, GPU-friendly, sample-based solver that can handle long-horizon tasks and high-dimensional state spaces, demonstrated on a Cartpole and a high-dimensional quadruped with real hardware. The approach enables complex mode compositions by integrating algorithmic controllers with learned policies (e.g., MPC/MPPI and learned feedback controllers) and has practical impact for enabling agile, stable, and reactive behaviors in legged robots and similar systems.

Abstract

This paper investigates a sample-based solution to the hybrid mode control problem across non-differentiable and algorithmic hybrid modes. Our approach reasons about a set of hybrid control modes as an integer-based optimization problem where we select what mode to apply, when to switch to another mode, and the duration for which we are in a given control mode. A sample-based variation is derived to efficiently search the integer domain for optimal solutions. We find our formulation yields strong performance guarantees that can be applied to a number of robotics-related tasks. In addition, our approach is able to synthesize complex algorithms and policies to compound behaviors and achieve challenging tasks. Last, we demonstrate the effectiveness of our approach in real-world robotic examples that require reactive switching between long-term planning and high-frequency control.

Sample-Based Hybrid Mode Control: Asymptotically Optimal Switching of Algorithmic and Non-Differentiable Control Modes

TL;DR

The paper tackles the problem of dynamically switching between discrete, potentially non-differentiable and algorithmic control modes in robotics by formulating an indefinite hybrid mode switching problem and introducing a discrete-time, sample-based optimization approach. It redefines the problem through a discrete-time horizon with decisions over the mode, start time, and duration , and then develops an iterative, sample-based method that draws uniform candidates from the single-mode transition set to achieve asymptotic convergence guarantees. Key contributions include a discrete-time definite formulation, an iterative sequencing scheme with local optimality properties, and a scalable, GPU-friendly, sample-based solver that can handle long-horizon tasks and high-dimensional state spaces, demonstrated on a Cartpole and a high-dimensional quadruped with real hardware. The approach enables complex mode compositions by integrating algorithmic controllers with learned policies (e.g., MPC/MPPI and learned feedback controllers) and has practical impact for enabling agile, stable, and reactive behaviors in legged robots and similar systems.

Abstract

This paper investigates a sample-based solution to the hybrid mode control problem across non-differentiable and algorithmic hybrid modes. Our approach reasons about a set of hybrid control modes as an integer-based optimization problem where we select what mode to apply, when to switch to another mode, and the duration for which we are in a given control mode. A sample-based variation is derived to efficiently search the integer domain for optimal solutions. We find our formulation yields strong performance guarantees that can be applied to a number of robotics-related tasks. In addition, our approach is able to synthesize complex algorithms and policies to compound behaviors and achieve challenging tasks. Last, we demonstrate the effectiveness of our approach in real-world robotic examples that require reactive switching between long-term planning and high-frequency control.
Paper Structure (14 sections, 3 theorems, 12 equations, 5 figures, 2 tables, 2 algorithms)

This paper contains 14 sections, 3 theorems, 12 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Hybrid Control
  4. Sample-based Control
  5. Problem Formulation: The (Indefinite) Hybrid Control Problem
  6. Sample-Based Hybrid Mode Control
  7. The Discrete (Definite) Hybrid Control Problem
  8. Iterative Hybrid Mode Control Sequencing
  9. Sample-Based Hybrid Mode Control
  10. Experiments and Results
  11. Scalability to High-dimensional Task
  12. Comparison to baseline control methods.
  13. Hardware Experiments
  14. Conclusions

Key Result

Proposition 1

(Locally Optimal Schedules) Let $f_m \forall m \in \mathcal{M}$ be deterministic, $\mathcal{K}^\star$ the optimal switching mode schedule, and $\mathcal{J}(\mathcal{K}^\star)$ a local optima. Then, $\mathcal{K}$ is optimal if $\forall \mu \in[0, T-1]$ and $\forall m \in \mathcal{M}$, $\nexists \nu \ $\mathcal{J}(\mathcal{K}) = \mathcal{J}(\mathcal{K}^\star)$, and $\mathcal{K}=\mathcal{K}^\star$ is

Figures (5)

  • Figure 1: Sample-Based Hybrid Mode Switching Enables Extreme Control Variations. We show the time-elapsed figure of the Unitree Go2 quadruped transitioning across three distinctive control modes (foot stand, jump flipping, and handstand) demonstrating that our method can achieve extremely agile motor skills within sharp transition times. The green, red, and yellow lines denote the head motion during each mode transitions.
  • Figure 2: Top Left: The optimal control mode sequences for the cartpole system, where the Y axis denotes the index for each corresponding sampled modes. Bottom Left: The resulting control applied to the actuators for the cartpole system are shown. The blue curve represents the control generated by the iLQR algorithm. The orange curve represents the executed control found using our sample-based hybrid scheduler, where the shaded areas denote the resulting distinguish modes. Right: The visualization of the cartpole trajectory with controls found by our method.
  • Figure 3: Top: Results for the short-horizon subproblem of cartpole system. The Y-axis quantifies the optimality gap, defined as the difference between the objective value and a locally optimal solution obtained via iterative LQR. To ensure scale invariance across tasks, we normalize this gap by the planning horizon length $H$. Here error-bar denotes standard deviation across $5$ random seeds. The performance of classical sampling methods deteriorates as the horizon length increases, whereas our method consistently finds good minima, even with a limited sample size. Bottom: Overall performance under a predictive control framework. Both methods show a decrease in cumulative cost as the horizon increases, as a longer horizon reduces myopic decision-making. However, as the horizon continues to grow, classical methods experience an increase in total cost due to their inability to effectively optimize the sub-problem, whereas our method continues to decrease the overall cost.
  • Figure 4: We verify our method on real Unitree Go2 quadruped. From left to right, the Unitree Go2 walks to autonomously avoid incoming obstacles. We update the control sequence in real time on a single CPU with onboard sensing, demonstrating the computation efficiency of our method.
  • Figure 5: State Visitation of Robot Pitch Rotation: We collect the trajectory robot motion and visualize the distribution of robot pitch angular position and velocity in the base frame. We show that our method can effective solving for control solutions that resides in distinct areas by optimizing control mode sequences.

Theorems & Definitions (9)

  • Definition 1
  • Definition 2
  • Definition 3
  • Proposition 1
  • proof
  • Theorem 1
  • proof
  • Theorem 2
  • proof