Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MM-LMPC: Multi-Modal Learning Model Predictive Control via Bandit-Based Mode Selection

Wataru Hashimoto, Kazumune Hashimoto

TL;DR

The paper tackles the limitation of Learning Model Predictive Control (LMPC) where exploration is biased toward early, low-cost trajectories, potentially missing globally superior solution modes. It proposes MM-LMPC, which clusters past trajectories into multiple modes, assigns mode-specific terminal sets and value functions, and uses a bandit-based meta-controller with a Lower Confidence Bound policy to select which mode to refine at each iteration. The authors prove recursive feasibility, stability, and asymptotic convergence to the best mode, along with a logarithmic regret bound $O(\log T)$ for mode exploration. A Dubins car reach-avoid simulation demonstrates that MM-LMPC systematically explores multiple routes and achieves lower final costs than standard LMPC, indicating improved global performance in iterative tasks.

Abstract

Learning Model Predictive Control (LMPC) improves performance on iterative tasks by leveraging data from previous executions. At each iteration, LMPC constructs a sampled safe set from past trajectories and uses it as a terminal constraint, with a terminal cost given by the corresponding cost-to-go. While effective, LMPC heavily depends on the initial trajectories: states with high cost-to-go are rarely selected as terminal candidates in later iterations, leaving parts of the state space unexplored and potentially missing better solutions. For example, in a reach-avoid task with two possible routes, LMPC may keep refining the initially shorter path while neglecting the alternative path that could lead to a globally better solution. To overcome this limitation, we propose Multi-Modal LMPC (MM-LMPC), which clusters past trajectories into modes and maintains mode-specific terminal sets and value functions. A bandit-based meta-controller with a Lower Confidence Bound (LCB) policy balances exploration and exploitation across modes, enabling systematic refinement of all modes. This allows MM-LMPC to escape high-cost local optima and discover globally superior solutions. We establish recursive feasibility, closed-loop stability, asymptotic convergence to the best mode, and a logarithmic regret bound. Simulations on obstacle-avoidance tasks validate the performance improvements of the proposed method.

MM-LMPC: Multi-Modal Learning Model Predictive Control via Bandit-Based Mode Selection

TL;DR

The paper tackles the limitation of Learning Model Predictive Control (LMPC) where exploration is biased toward early, low-cost trajectories, potentially missing globally superior solution modes. It proposes MM-LMPC, which clusters past trajectories into multiple modes, assigns mode-specific terminal sets and value functions, and uses a bandit-based meta-controller with a Lower Confidence Bound policy to select which mode to refine at each iteration. The authors prove recursive feasibility, stability, and asymptotic convergence to the best mode, along with a logarithmic regret bound for mode exploration. A Dubins car reach-avoid simulation demonstrates that MM-LMPC systematically explores multiple routes and achieves lower final costs than standard LMPC, indicating improved global performance in iterative tasks.

Abstract

Learning Model Predictive Control (LMPC) improves performance on iterative tasks by leveraging data from previous executions. At each iteration, LMPC constructs a sampled safe set from past trajectories and uses it as a terminal constraint, with a terminal cost given by the corresponding cost-to-go. While effective, LMPC heavily depends on the initial trajectories: states with high cost-to-go are rarely selected as terminal candidates in later iterations, leaving parts of the state space unexplored and potentially missing better solutions. For example, in a reach-avoid task with two possible routes, LMPC may keep refining the initially shorter path while neglecting the alternative path that could lead to a globally better solution. To overcome this limitation, we propose Multi-Modal LMPC (MM-LMPC), which clusters past trajectories into modes and maintains mode-specific terminal sets and value functions. A bandit-based meta-controller with a Lower Confidence Bound (LCB) policy balances exploration and exploitation across modes, enabling systematic refinement of all modes. This allows MM-LMPC to escape high-cost local optima and discover globally superior solutions. We establish recursive feasibility, closed-loop stability, asymptotic convergence to the best mode, and a logarithmic regret bound. Simulations on obstacle-avoidance tasks validate the performance improvements of the proposed method.

Paper Structure

This paper contains 13 sections, 5 theorems, 32 equations, 4 figures, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. Problem Formulation
  3. Review of Learning Model Predictive Control (LMPC)
  4. Proposed Methodology: Multi-Modal Learning Model Predictive Control (MM-LMPC)
  5. Mode Clustering
  6. Mode-Specific LMPC
  7. Meta-Controller for Mode Selection via Multi-Armed Bandits
  8. Summary of the Proposed Algorithm
  9. Theoretical Analysis
  10. Simulation Study
  11. Experimental Setup
  12. Results
  13. Conclusion

Key Result

Theorem 1

Under Assumptions assum:system--assum:classifier_consistency, the MM-LMPC controller is recursively feasible for all iterations $j \geq 1$ and time steps $t \geq 0$. Furthermore, for each fixed iteration $j$, the closed-loop system is asymptotically stable.

Figures (4)

  • Figure 1: Execution example of standard LMPC: black dashed/dotted are initial seeds, gray are rollouts, bold curve is the final best. Obstacle shown in red.
  • Figure 2: The proposed MM-LMPC architecture.
  • Figure 3: The trajectories obtained at the last iteration (blue: MM-LMPC, orange dashed: LMPC). Red ellipse: obstacle.
  • Figure 4: MM-LMPC trajectories during learning (gray), initial trajectories (black dashed/dotted), and the final best path (blue). Red ellipse: obstacle.

Theorems & Definitions (10)

  • Theorem 1: Recursive Feasibility and Stability
  • proof
  • Lemma 1: Intra-Mode Non-Increasing Cost
  • proof
  • Theorem 2: Asymptotic Optimality
  • proof
  • Theorem 3: Bound on Iteration Cost under LCB Selection
  • proof
  • Theorem 4: Logarithmic Regret Bound
  • proof