Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentiable Nonlinear Model Predictive Control

Jonathan Frey, Katrin Baumgärtner, Gianluca Frison, Dirk Reinhardt, Jasper Hoffmann, Leonard Fichtner, Sebastien Gros, Moritz Diehl

TL;DR

The paper tackles the need for reliable parametric solution sensitivities in learning-augmented nonlinear MPC by formulating a differentiable solver based on interior-point methods and the implicit function theorem. It develops a two-layer approach that uses smoothed KKT conditions to produce differentiable solution mappings and forwards/adjoints, implemented efficiently in acados for general OCPs. The results show substantial CPU-time speedups over leading ML-oriented solvers and demonstrate feasibility for high-dimensional, highly-parametric problems, enabling gradient-based learning within MPC. This work paves the way for integrating differentiable MPC as a learning-friendly module in real-time control systems.

Abstract

The efficient computation of parametric solution sensitivities is a key challenge in the integration of learning-enhanced methods with nonlinear model predictive control (MPC), as their availability is crucial for many learning algorithms. This paper discusses the computation of solution sensitivities of general nonlinear programs (NLPs) using the implicit function theorem (IFT) and smoothed optimality conditions treated in interior-point methods (IPM). We detail sensitivity computation within a sequential quadratic programming (SQP) method which employs an IPM for the quadratic subproblems. Previous works presented in the machine learning community are limited to convex or unconstrained formulations, or lack an implementation for efficient sensitivity evaluation. The publication is accompanied by an efficient open-source implementation within the acados framework, providing both forward and adjoint sensitivities for general optimal control problems, achieving speedups exceeding 3x over the state-of-the-art solvers mpc.pytorch and cvxpygen.

Differentiable Nonlinear Model Predictive Control

TL;DR

The paper tackles the need for reliable parametric solution sensitivities in learning-augmented nonlinear MPC by formulating a differentiable solver based on interior-point methods and the implicit function theorem. It develops a two-layer approach that uses smoothed KKT conditions to produce differentiable solution mappings and forwards/adjoints, implemented efficiently in acados for general OCPs. The results show substantial CPU-time speedups over leading ML-oriented solvers and demonstrate feasibility for high-dimensional, highly-parametric problems, enabling gradient-based learning within MPC. This work paves the way for integrating differentiable MPC as a learning-friendly module in real-time control systems.

Abstract

The efficient computation of parametric solution sensitivities is a key challenge in the integration of learning-enhanced methods with nonlinear model predictive control (MPC), as their availability is crucial for many learning algorithms. This paper discusses the computation of solution sensitivities of general nonlinear programs (NLPs) using the implicit function theorem (IFT) and smoothed optimality conditions treated in interior-point methods (IPM). We detail sensitivity computation within a sequential quadratic programming (SQP) method which employs an IPM for the quadratic subproblems. Previous works presented in the machine learning community are limited to convex or unconstrained formulations, or lack an implementation for efficient sensitivity evaluation. The publication is accompanied by an efficient open-source implementation within the acados framework, providing both forward and adjoint sensitivities for general optimal control problems, achieving speedups exceeding 3x over the state-of-the-art solvers mpc.pytorch and cvxpygen.
Paper Structure (39 sections, 4 theorems, 28 equations, 5 figures, 2 tables)

This paper contains 39 sections, 4 theorems, 28 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contribution.
  3. Related work.
  4. Outline.
  5. Preliminaries
  6. Solving nonlinear programs with an interior-point based SQP method
  7. Sequential quadratic programming.
  8. QP solution via interior-point methods.
  9. Parametric solution sensitivities within an IP-based SQP method
  10. Forward and adjoint sensitivity computation.
  11. Computational complexity.
  12. Differentiable MPC
  13. Structure exploiting solvers.
  14. Efficient solution and sensitivity computation.
  15. Efficient implementation in acados
  16. ...and 24 more sections

Key Result

Theorem 1

Suppose Assumption as:regularity holds at a KKT point $z^\star$ of eq:z_star_def with parameter $\bar{\theta}$. In a neighborhood of $\bar{\theta}$, there exists a differentiable function $z^\mathrm{sol}(\theta)$ with $z^\mathrm{sol}(\bar{\theta}) = z^\star$ that corresponds to a locally unique solu

Figures (5)

  • Figure 1: Simple example visualizing a nonsmooth solution map, its derivative and some smooth approximations.
  • Figure 2: Optimal solution $u_0$ and sensitivities obtained by acados compared with finite differences for the OCP described in Sec. \ref{['sec:solution_sens_pendulum']}. At active set changes, which are indicated by multipliers switching from being zero to being positive (or vice versa), the solution map is nondifferentiable.
  • Figure 3: Computation time for evaluating different kinds of sensitivities on the example in Sec. \ref{['sec:chain_sens']}.
  • Figure 4: Example of an NLP \ref{['eq:jump_ocp']} for which the global solution $x^\star(\theta)$ jumps. The objective function of \ref{['eq:jump_ocp']} is visualized for different values of $\theta$ on the left. The plots on the right show the numerical solution and corresponding objective function value obtained with different solver initializations $x{_{\mathrm{init}}}$.
  • Figure 5: Schematic overview of the class hierarchy for integrating the proposed approach in common ML frameworks via AcadosDiffMpcFunction and specifically for PyTorch via AcadosDiffMpcTorch.

Theorems & Definitions (9)

  • Example 1
  • Remark 1: Hessian approximations
  • Theorem 1: NLP solution sensitivity existence
  • Theorem 2: NLP and QP sensitivities coincide
  • Theorem 3: Smoothed KKT system
  • Remark 2: Differentiating accross active set changes
  • Remark 3: Approximate Hessian pitfall
  • Remark 4: Riccati variants & regularity
  • Theorem 4: Implicit function theorem (IFT)