Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging the Sim-to-real Gap: A Control Framework for Imitation Learning of Model Predictive Control

Seungtaek Kim, Jonghyup Lee, Kyoungseok Han, Seibum B. Choi

Abstract

To address the computational challenges of Model Predictive Control (MPC), recent research has studied using imitation learning to approximate MPC with a computationally efficient Deep Neural Network (DNN). However, this introduces a common issue in learning-based control, the simulation-to-reality (sim-to-real) gap. Inspired by Robust Tube MPC, this study proposes a new control framework that addresses this issue from a control perspective. The framework ensures the DNN operates in the same environment as the source domain, addressing the sim-to-real gap with great data collection efficiency. Moreover, an input refinement governor is introduced to address the DNN's inability to adapt to variations in model parameters, enabling the system to satisfy MPC constraints more robustly under parameter-changing conditions. The proposed framework was validated through two case studies: cart-pole control and vehicle collision avoidance control, which analyzed the principles of the proposed framework in detail and demonstrated its application to a vehicle control case.

Bridging the Sim-to-real Gap: A Control Framework for Imitation Learning of Model Predictive Control

Abstract

To address the computational challenges of Model Predictive Control (MPC), recent research has studied using imitation learning to approximate MPC with a computationally efficient Deep Neural Network (DNN). However, this introduces a common issue in learning-based control, the simulation-to-reality (sim-to-real) gap. Inspired by Robust Tube MPC, this study proposes a new control framework that addresses this issue from a control perspective. The framework ensures the DNN operates in the same environment as the source domain, addressing the sim-to-real gap with great data collection efficiency. Moreover, an input refinement governor is introduced to address the DNN's inability to adapt to variations in model parameters, enabling the system to satisfy MPC constraints more robustly under parameter-changing conditions. The proposed framework was validated through two case studies: cart-pole control and vehicle collision avoidance control, which analyzed the principles of the proposed framework in detail and demonstrated its application to a vehicle control case.

Paper Structure

This paper contains 20 sections, 2 theorems, 60 equations, 13 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. Notation
  3. PROBLEM STATEMENT
  4. Imitation learning of model predictive control
  5. Sim-to-real gap
  6. PROPOSED CONTROL FRAMEWORK
  7. Brief review of RTMPC
  8. Nominal and ancillary controllers
  9. Remark
  10. Input refinement governor
  11. Remark
  12. CASE STUDY
  13. Cart-pole system
  14. Proposed control framework
  15. Results: sim-to-real gap
  16. ...and 5 more sections

Key Result

Proposition 1

If the DNN $\pi_\theta$, approximating the MPC, is composed as a nominal controller in the overall control structure attached with an ancillary controller as follows: where $\bar{x}$ is the nominal state propagated by the nominal model, and $x$ is the actual state, then the target domain of the DNN controller becomes equal to the nominal model-based domain $\mathcal{S_\mathrm{nom}}$.

Figures (13)

  • Figure 1: RTMPC concept.
  • Figure 2: Diagrams representing (a) Domain Randomization (DR) and (b) the proposed framework.
  • Figure 3: The overall structure of the proposed control framework.
  • Figure 4: Cart-pole system.
  • Figure 5: Controlled cart position and the control inputs in the target domain.
  • ...and 8 more figures

Theorems & Definitions (4)

  • Definition 1: Nominal Model-based Domain $\mathcal{S_\mathrm{nom}}$
  • Proposition 1
  • Definition 2: Input refinement governor
  • Proposition 2