Faster Model Predictive Control via Self-Supervised Initialization Learning

TL;DR

MPC provides optimal control over a horizon but can be too slow for real-time deployment. This paper presents a two-phase warm-start framework that first learns a high-quality initial guess via offline behavior cloning and then online reinforcement learning with DAgger to minimize MPC runtime while maintaining performance, applicable to both COBYLA-based deterministic MPC and MPPI-based sampling MPC. Empirical results show up to ~$21.6\%$ faster optimization and ~$34.1\%$ better tracking for deterministic MPC, and substantial safety and efficiency gains for MPPI in obstacle-rich scenarios, with strong zero-shot generalization. By preserving the MPC formulation and only improving initialization, the approach offers practical speedups and robustness, and can extend to a broader class of optimization-based controllers beyond MPC.

Abstract

Model Predictive Control (MPC) is widely used in robot control by optimizing a sequence of control outputs over a finite-horizon. Computational approaches for MPC include deterministic methods (e.g., iLQR and COBYLA), as well as sampling-based methods (e.g., MPPI and CEM). However, complex system dynamics and non-convex or non-differentiable cost terms often lead to prohibitive optimization times that limit real-world deployment. Prior efforts to accelerate MPC have limitations on: (i) reusing previous solutions fails under sharp state changes and (ii) pure imitation learning does not target compute efficiency directly and suffers from suboptimality in the training data. To address these, We propose a warm-start framework that learns a policy to generate high-quality initial guesses for MPC solver. The policy is first trained via behavior cloning from expert MPC rollouts and then fine-tuned online with reinforcement learning to directly minimize MPC optimization time. We empirically validate that our approach improves both deterministic and sampling-based MPC methods, achieving up to 21.6% faster optimization and 34.1% more tracking accuracy for deterministic MPC in Formula 1 track path-tracking domain, and improving safety by 100%, path efficiency by 12.8%, and steering smoothness by 7.2% for sampling-based MPC in obstacle-rich navigation domain. These results demonstrate that our framework not only accelerates MPC but also improves overall control performance. Furthermore, it can be applied to a broader range of control algorithms that benefit from good initial guesses.

Figures (4)

• Figure 1: Overview of our proposed algorithm. The first two blocks denote the two-phase training framework. In the first phase, we collect expert MPC demonstrations and train a warm-start policy using behavior cloning to speed up MPC. In the second phase, we fine-tune this policy within an online training framework to enhance its performance and generalizability. During testing, the proposed framework is evaluated on both training tracks and challenging zero-shot tracks, as demonstrated in the third block.
• Figure 2: Training and Testing maps used in the Formula 1 track path-tracking domain.
• Figure 3: Training and testing maps in the obstacle-rich navigation domain. Green dots indicate spawn points, red dots represent destinations, and polygons correspond to obstacles.
• Figure 4: Visualization of optimized trajectories for our method and the baselines on a segment of the Nürburgring map.