Control-Theoretic Techniques for Online Adaptation of Deep Neural Networks in Dynamical Systems

TL;DR

This work addresses the lack of intrinsic stability guarantees for online adaptation of deep neural networks (DNNs) in dynamical systems under domain shift. It treats a DNN as a continuous-time dynamical system and uses control-theoretic online updates—specifically a last-layer update governed by the Super-Twisting Algorithm ($STA$)—to achieve provable convergence of the prediction error. Spectral normalization is incorporated during training to bound the Lipschitz constant, enabling rigorous control of error trajectories when input derivatives are noisy. The approach yields two main results: Case I, where $\dot{x}$ is known, guarantees robust finite-time convergence of the output error; Case II, where $\dot{x}$ is unknown or estimated, guarantees ultimate boundedness with disturbance bounds tied to the Lipschitz constant. Simulations on the Van der Pol oscillator under domain shift validate the theoretical guarantees and demonstrate improved resilience to model mismatch and measurement noise, highlighting the practical impact for sim2real and online forecasting tasks.

Abstract

Deep neural networks (DNNs), trained with gradient-based optimization and backpropagation, are currently the primary tool in modern artificial intelligence, machine learning, and data science. In many applications, DNNs are trained offline, through supervised learning or reinforcement learning, and deployed online for inference. However, training DNNs with standard backpropagation and gradient-based optimization gives no intrinsic performance guarantees or bounds on the DNN, which is essential for applications such as controls. Additionally, many offline-training and online-inference problems, such as sim2real transfer of reinforcement learning policies, experience domain shift from the training distribution to the real-world distribution. To address these stability and transfer learning issues, we propose using techniques from control theory to update DNN parameters online. We formulate the fully-connected feedforward DNN as a continuous-time dynamical system, and we propose novel last-layer update laws that guarantee desirable error convergence under various conditions on the time derivative of the DNN input vector. We further show that training the DNN under spectral normalization controls the upper bound of the error trajectories of the online DNN predictions, which is desirable when numerically differentiated quantities or noisy state measurements are input to the DNN. The proposed online DNN adaptation laws are validated in simulation to learn the dynamics of the Van der Pol system under domain shift, where parameters are varied in inference from the training dataset. The simulations demonstrate the effectiveness of using control-theoretic techniques to derive performance improvements and guarantees in DNN-based learning systems.

Figures (7)

• Figure 1: DNN predictions on the nominal system ($\epsilon = 1$).
• Figure 2: DNN predictions on the real system ($\epsilon = 1.5$).
• Figure 3: Online-adapted DNN predictions on the real system ($\epsilon = 1.5$).
• Figure 4: Block diagram of the proposed online learning method, for a sim2real control example. The neural network model is first trained offline using conventional supervised or reinforcement learning under spectral normalization. The learned model, once deployed, is then updated online using the adaptation laws in Equations \ref{['eq:b_update_st']}, \ref{['eq:W_update_st']}, and \ref{['eq:z_update_st']}.
• Figure 5: Online-adapted DNN predictions on the real system ($\epsilon = 1.5$) for each case.

Theorems & Definitions (6)

• Theorem 5.1
• proof
• Lemma 5.2
• proof
• Theorem 5.3
• proof