Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MPC of Uncertain Nonlinear Systems with Meta-Learning for Fast Adaptation of Neural Predictive Models

Jiaqi Yan, Ankush Chakrabarty, Alisa Rupenyan, John Lygeros

TL;DR

The paper tackles reference tracking for unknown nonlinear systems by introducing a Neural State-Space Model (NSSM) that lifts nonlinear dynamics into a latent linear space, enabling model predictive control (MPC). It couples NSSM with an iMAML-based meta-learning framework to pre-train on source systems and rapidly adapt to a target system using limited data, leveraging the implicit-function gradient to avoid storing full inner-loop optimization paths. Empirical results on Van der Pol oscillators and a pendulum demonstrate that the iMAML-based NSSM yields more accurate predictions and faster, more robust tracking under MPC than MAML-based or supervised baselines. This data-efficient approach holds practical significance for robotics and manufacturing, where collecting target-system data is costly, by enabling fast adaptation with high control performance.

Abstract

In this paper, we consider the problem of reference tracking in uncertain nonlinear systems. A neural State-Space Model (NSSM) is used to approximate the nonlinear system, where a deep encoder network learns the nonlinearity from data, and a state-space component captures the temporal relationship. This transforms the nonlinear system into a linear system in a latent space, enabling the application of model predictive control (MPC) to determine effective control actions. Our objective is to design the optimal controller using limited data from the \textit{target system} (the system of interest). To this end, we employ an implicit model-agnostic meta-learning (iMAML) framework that leverages information from \textit{source systems} (systems that share similarities with the target system) to expedite training in the target system and enhance its control performance. The framework consists of two phases: the (offine) meta-training phase learns a aggregated NSSM using data from source systems, and the (online) meta-inference phase quickly adapts this aggregated model to the target system using only a few data points and few online training iterations, based on local loss function gradients. The iMAML algorithm exploits the implicit function theorem to exactly compute the gradient during training, without relying on the entire optimization path. By focusing solely on the optimal solution, rather than the path, we can meta-train with less storage complexity and fewer approximations than other contemporary meta-learning algorithms. We demonstrate through numerical examples that our proposed method can yield accurate predictive models by adaptation, resulting in a downstream MPC that outperforms several baselines.

MPC of Uncertain Nonlinear Systems with Meta-Learning for Fast Adaptation of Neural Predictive Models

TL;DR

The paper tackles reference tracking for unknown nonlinear systems by introducing a Neural State-Space Model (NSSM) that lifts nonlinear dynamics into a latent linear space, enabling model predictive control (MPC). It couples NSSM with an iMAML-based meta-learning framework to pre-train on source systems and rapidly adapt to a target system using limited data, leveraging the implicit-function gradient to avoid storing full inner-loop optimization paths. Empirical results on Van der Pol oscillators and a pendulum demonstrate that the iMAML-based NSSM yields more accurate predictions and faster, more robust tracking under MPC than MAML-based or supervised baselines. This data-efficient approach holds practical significance for robotics and manufacturing, where collecting target-system data is costly, by enabling fast adaptation with high control performance.

Abstract

In this paper, we consider the problem of reference tracking in uncertain nonlinear systems. A neural State-Space Model (NSSM) is used to approximate the nonlinear system, where a deep encoder network learns the nonlinearity from data, and a state-space component captures the temporal relationship. This transforms the nonlinear system into a linear system in a latent space, enabling the application of model predictive control (MPC) to determine effective control actions. Our objective is to design the optimal controller using limited data from the \textit{target system} (the system of interest). To this end, we employ an implicit model-agnostic meta-learning (iMAML) framework that leverages information from \textit{source systems} (systems that share similarities with the target system) to expedite training in the target system and enhance its control performance. The framework consists of two phases: the (offine) meta-training phase learns a aggregated NSSM using data from source systems, and the (online) meta-inference phase quickly adapts this aggregated model to the target system using only a few data points and few online training iterations, based on local loss function gradients. The iMAML algorithm exploits the implicit function theorem to exactly compute the gradient during training, without relying on the entire optimization path. By focusing solely on the optimal solution, rather than the path, we can meta-train with less storage complexity and fewer approximations than other contemporary meta-learning algorithms. We demonstrate through numerical examples that our proposed method can yield accurate predictive models by adaptation, resulting in a downstream MPC that outperforms several baselines.
Paper Structure (14 sections, 1 theorem, 29 equations, 6 figures, 2 algorithms)

This paper contains 14 sections, 1 theorem, 29 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Neural State-Space Model (NSSM)
  4. NSSM-enabled MPC
  5. Meta-learning framework
  6. Meta-Learning-Based MPC Design
  7. Bi-level optimization problem in meta-learning
  8. Approximation solutions to the bi-level optimization problem
  9. Meta-inference
  10. Comparison with MAML
  11. Simulation
  12. Van der Pol oscillators
  13. Pendulum
  14. Conclusion

Key Result

Lemma 1

Define where $\nabla_{\psi}^2 \ell_{\text{SSM}}(\mathcal{D}^b_{\text{tr}} ;\psi)|_{\psi=\omega^b}$ is value of the Hessian matrix $\nabla_{\psi}^2 \ell_{\text{SSM}}(\mathcal{D}^b_{\text{tr}} ;\psi)$ evaluated at $\psi=\omega^b$. If $Q^b$ is invertible, then

Figures (6)

  • Figure 1: Information flow in the meta-training phase.
  • Figure 2: Comparison of the prediction performance in the meta-training phase. The lines and shaded regions respectively represent the mean and standard deviation across $10$ runs.
  • Figure 3: Comparison of the prediction performance on the target system.
  • Figure 4: The Pendulum Gym Environment.
  • Figure 5: Comparison of the tracking performance on the target system, where the three columns show the performance of different algorithms by using the NSSMs adapted after $10$, $100$, and $3000$ steps, respectively. The lines and shaded regions respectively represent the mean and standard deviation across $10$ runs.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Lemma 1: rajeswaran2019meta