Algorithmic design and implementation considerations of deep MPC

TL;DR

This work investigates Deep Model Predictive Control (Deep MPC), where a deep neural network in the loop learns unknown disturbances online while an MPC layer enforces state and input constraints. The proposed architecture combines a reference governor, a tracking MPC, an adaptation mechanism, and two bounded-output neural networks, distributing control between a learning term $u_t^a$ and an MPC term $u_t^m$ under a bound $\\|u_t\|_\infty \le u_{\max}$. A key contribution is the algorithmic set of procedures for constraint tightening, online outer-layer weight updates with projection, and replay-buffer–driven transfer learning to update inner features, ensuring the learned component remains bounded and stable. A skid-steer robot numerical experiment illustrates that insufficient learning authority (small $u_{\max}^a$) saturates the learning signal and yields performance close to tube-MPC, while appropriate authority enables meaningful learning and improved performance.

Abstract

Deep Model Predictive Control (Deep MPC) is an evolving field that integrates model predictive control and deep learning. This manuscript is focused on a particular approach, which employs deep neural network in the loop with MPC. This class of approaches distributes control authority between a neural network and an MPC controller, in such a way that the neural network learns the model uncertainties while the MPC handles constraints. The approach is appealing because training data collected while the system is in operation can be used to fine-tune the neural network, and MPC prevents unsafe behavior during those learning transients. This manuscript explains implementation challenges of Deep MPC, algorithmic way to distribute control authority and argues that a poor choice in distributing control authority may lead to poor performance. A reason of poor performance is explained through a numerical experiment on a four-wheeled skid-steer dynamics.

Figures (4)

• Figure 1: Deep MPC has a neural network in the loop with MPC in which the outermost layer is adapted at each time instant and its output is stored in a replay buffer to create training dataset to train its inner layers intermittently in a remote training thread.
• Figure 2: The neural network architecture used in Deep MPC.
• Figure 3: In the absence of enough learning authority, performance of Deep MPC is close to that of tube MPC.
• Figure 4: In the absence of enough learning authority, $u^a$ become saturated and therefore, $\tilde{u} = h+u^a$ changes according to $h$.