Optimal Control Theoretic Neural Optimizer: From Backpropagation to Dynamic Programming

TL;DR

The paper links DNN training to optimal control by viewing the forward pass as a dynamical system and showing Backpropagation corresponds to a first-order dynamic programming update of the Bellman objective $Q_k(x_k, theta_k)$. It then introduces Optimal Control Theoretic Neural Optimizer (OCNOpt), a second-order extension inspired by Differential Dynamic Programming (DDP) and implemented with a Gauss-Newton-based, low-rank factorization to keep computational costs practical. OCNOpt enables layer-wise feedback policies and can be extended to Residual networks, Neural ODEs, and dynamic games, achieving improved robustness and faster convergence across image classification, time-series, and probabilistic modeling tasks while maintaining manageable overhead. This principled, control-theoretic approach opens new avenues for algorithm design in deep learning grounded in dynamical systems and optimal control theory.

Abstract

Optimization of deep neural networks (DNNs) has been a driving force in the advancement of modern machine learning and artificial intelligence. With DNNs characterized by a prolonged sequence of nonlinear propagation, determining their optimal parameters given an objective naturally fits within the framework of Optimal Control Programming. Such an interpretation of DNNs as dynamical systems has proven crucial in offering a theoretical foundation for principled analysis from numerical equations to physics. In parallel to these theoretical pursuits, this paper focuses on an algorithmic perspective. Our motivated observation is the striking algorithmic resemblance between the Backpropagation algorithm for computing gradients in DNNs and the optimality conditions for dynamical systems, expressed through another backward process known as dynamic programming. Consolidating this connection, where Backpropagation admits a variational structure, solving an approximate dynamic programming up to the first-order expansion leads to a new class of optimization methods exploring higher-order expansions of the Bellman equation. The resulting optimizer, termed Optimal Control Theoretic Neural Optimizer (OCNOpt), enables rich algorithmic opportunities, including layer-wise feedback policies, game-theoretic applications, and higher-order training of continuous-time models such as Neural ODEs. Extensive experiments demonstrate that OCNOpt improves upon existing methods in robustness and efficiency while maintaining manageable computational complexity, paving new avenues for principled algorithmic design grounded in dynamical systems and optimal control theory.

Figures (13)

• Figure 1: We propose a new class of training optimizers for DNNs, termed Optimal Control Theoretic Neural Optmizer (OCNOpt), that interpret the forward pass as dynamical system propagation and, from which, explore the variational structure of Backpropagation as solving dynamic programming.
• Figure 2: Example of a residual block as $F_k(x_k, \theta_k, \phi_k)$ in (\ref{['F']}).
• Figure 3: Computational complexity between various optimization methods on training (left) a CNN on MNIST dataset and (right) a ResNet18 on CIFAR100 with respect to batch sizes. Notice that all axes use log scale.
• Figure 4: (a) Relative computational complexity of our OCNOpt compared to the first-order method Adam, which are donoted by the black dashed lines. We shorthand, e.g., "Mn" as MNIST, "Sv" as SVHN, and so on. (b) Convergence as a function of wall-clock runtime between our OCNOpt and other first-order methods, including Adam and SGD. Note that the "k" in the x-axis abbreviates "$1000$".
• Figure 5: Accuracy (%) improvement (+) or degradation (-) observed when layer-wise feedback policies are enabled for each best-tuned baseline from Table \ref{['tb:ddp-clf']}. Color bar is scaled for best view.

Theorems & Definitions (17)

• Theorem 1
• Corollary 1
• Proposition 1
• Corollary 1
• Proposition 1
• Theorem 1
• proof
• Corollary 1
• proof
• Lemma 2