When an Approximate Model Suffices for Optimal Control

TL;DR

The paper addresses when a control policy synthesized from an approximate model remains optimal for the actual system under model mismatch. It introduces a penalized model based optimal control framework and establishes that, under mild convexity and regularity conditions, the pointwise minimizers of the model based and plant based Hamiltonians coincide, leading to identical control trajectories over a finite horizon. A special case with quadratic control effort yields explicit, easily verifiable sufficiency conditions for equivalence and uniqueness. Numerical examples with significant model mismatch demonstrate that equivalence can hold even when state trajectories differ, supporting the practical viability of digital twins for robust model based decision making. The results provide a principled, continuous time explanation for the observed success of learning aided control architectures and suggest focusing learning on regimes where Hamiltonian minimizers align rather than achieving exact model fidelity.

Abstract

In this paper, we develop an optimal control framework for dynamical systems when only an approximate model of the underlying plant is available. We consider a setting in which the control strategy is synthesized using a model-based optimal control problem that includes a penalty term capturing deviation from the plant trajectory, while the same control input is applied to both the model and the actual system. For a general class of optimal control problems, we establish conditions under which the control minimizing the model-based Hamiltonian coincides with the plant-optimal control, despite mismatch between the model and the true dynamics. We further specialize these results to problems with quadratic control effort, where explicit and easily verifiable sufficient conditions guarantee equivalence and uniqueness of the resulting optimal control. These results show that accurate control synthesis does not require an exact model of the underlying system, but rather alignment of the optimality conditions that govern control selection. From a learning perspective, this suggests that data-driven efforts can focus on identifying regimes in which model-based and plant-based Hamiltonian minimizers coincide, thereby providing a theoretical basis for robust model-based decision making and the effective use of digital twins under modeling error. We provide examples to illustrate the theoretical findings and demonstrate equivalence of the resulting control trajectories even in the presence of significant model mismatch.

Figures (3)

• Figure 1: Constrained optimal controls $u^*(t)$ (plant) and $u^\circ(t)$ (model). This figure illustrates the equivalence of constrained Hamiltonian minimizers stated in Theorem 3.
• Figure 2: Unconstrained Hamiltonian minimizers for the plant and model, together with control bounds. The unconstrained minimizers differ due to model mismatch, but both lie far outside $\mathcal{U}$ for all $t$.
• Figure 3: State trajectories of the plant and the model under the identical optimal control. The trajectories differ due to mismatched dynamics, even though the control is the same.

Theorems & Definitions (11)

• Theorem 1: Existence and uniqueness
• proof
• Remark 1: First-order optimality in variational inequality form
• Theorem 2
• proof
• Remark 2
• Lemma 1: Existence and uniqueness under quadratic effort
• proof
• Theorem 3: Structural equivalence of optimal controls
• proof