Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Universal Approximation of Dynamical Systems by Semi-Autonomous Neural ODEs and Applications

Ziqian Li, Kang Liu, Lorenzo Liverani, Enrique Zuazua

TL;DR

The paper introduces semi-autonomous neural ODEs (SA-NODEs) as a parameter-efficient variant of neural ODEs and proves a universal approximation property for dynamical systems on a finite horizon, with the error decaying as the width $P$ grows. It provides a quantitative rate $O(P^{-1/2})$ under Sobolev regularity via Barron-space embeddings and establishes a parallel $O(P^{-1/2})$ rate for transport equations in Wasserstein-1 distance, improving upon prior transport-ODE results. A training framework based on optimal-control with adjoint gradient is developed, and SA-NODEs are validated through numerical experiments on ODEs and 2D transport equations, showing improved accuracy and reduced complexity versus vanilla NODEs. Overall, SA-NODEs offer a robust, scalable approach for data-driven learning of continuous-time dynamics and transport processes, with clear advantages in stability and efficiency.

Abstract

In this paper, we introduce semi-autonomous neural ordinary differential equations (SA-NODEs), a variation of the vanilla NODEs, employing fewer parameters. We investigate the universal approximation properties of SA-NODEs for dynamical systems from both a theoretical and a numerical perspective. Within the assumption of a finite-time horizon, under general hypotheses we establish an asymptotic approximation result, demonstrating that the error vanishes as the number of parameters goes to infinity. Under additional regularity assumptions, we further specify this convergence rate in relation to the number of parameters, utilizing quantitative approximation results in the Barron space. Based on the previous result, we prove an approximation rate for transport equations by their neural counterparts. Our numerical experiments validate the effectiveness of SA-NODEs in capturing the dynamics of various ODE systems and transport equations. Additionally, we compare SA-NODEs with vanilla NODEs, highlighting the superior performance and reduced complexity of our approach.

Universal Approximation of Dynamical Systems by Semi-Autonomous Neural ODEs and Applications

TL;DR

The paper introduces semi-autonomous neural ODEs (SA-NODEs) as a parameter-efficient variant of neural ODEs and proves a universal approximation property for dynamical systems on a finite horizon, with the error decaying as the width grows. It provides a quantitative rate under Sobolev regularity via Barron-space embeddings and establishes a parallel rate for transport equations in Wasserstein-1 distance, improving upon prior transport-ODE results. A training framework based on optimal-control with adjoint gradient is developed, and SA-NODEs are validated through numerical experiments on ODEs and 2D transport equations, showing improved accuracy and reduced complexity versus vanilla NODEs. Overall, SA-NODEs offer a robust, scalable approach for data-driven learning of continuous-time dynamics and transport processes, with clear advantages in stability and efficiency.

Abstract

In this paper, we introduce semi-autonomous neural ordinary differential equations (SA-NODEs), a variation of the vanilla NODEs, employing fewer parameters. We investigate the universal approximation properties of SA-NODEs for dynamical systems from both a theoretical and a numerical perspective. Within the assumption of a finite-time horizon, under general hypotheses we establish an asymptotic approximation result, demonstrating that the error vanishes as the number of parameters goes to infinity. Under additional regularity assumptions, we further specify this convergence rate in relation to the number of parameters, utilizing quantitative approximation results in the Barron space. Based on the previous result, we prove an approximation rate for transport equations by their neural counterparts. Our numerical experiments validate the effectiveness of SA-NODEs in capturing the dynamics of various ODE systems and transport equations. Additionally, we compare SA-NODEs with vanilla NODEs, highlighting the superior performance and reduced complexity of our approach.
Paper Structure (24 sections, 9 theorems, 87 equations, 8 figures, 1 table)

This paper contains 24 sections, 9 theorems, 87 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Neural ODEs
  3. Main results
  4. Methodology
  5. Qualitative convergence
  6. Quantitative convergence
  7. Transport equation
  8. Related works
  9. Outline of the Paper
  10. Main results
  11. Notations
  12. Semi-Autonomous Neural ODE
  13. Main results
  14. Proof of main results
  15. Proof of Theorem \ref{['th:UAP']}
  16. ...and 9 more sections

Key Result

Theorem 2.1

Let Assumption ass1 hold true. For any compact set $K\subseteq \mathbb{R}^{d}$ and any $\varepsilon>0$, there exists a constant $P_{\varepsilon, T, K, f}$ such that for any $P\geq P_{\varepsilon, T, K, f}$, there exist parameters $(W_i,A^1_i,A^2_i,B_i) \in \mathbb{R}^d \times \mathbb{R}^{d\times d} where $\boldsymbol{z}_{z_0}(\cdot)$ (resp. $\boldsymbol{x}_{z_0}(\cdot)$) is the solution of eq:ode

Figures (8)

  • Figure 5.1: SA-NODEs solution, exact solution and errors of ODE systems.
  • Figure 5.2: Comparison of vanilla NODEs and SA-NODEs on solutions and errors for system \ref{['eq:exam_ODE_auto']}.
  • Figure 5.3: Comparison of vanilla NODEs and SA-NODEs on solutions and errors for system \ref{['eq:exam_ODE_nonauto']}.
  • Figure 5.4: Comparison of test errors for vanilla NODEs and SA-NODEs on system \ref{['eq:exam_ODE_nonauto']}: (Left) Training set size fixed at 220 trajectories, number of training epochs varies from $10$ to $10^4$; (Right) Number of training epochs fixed at $10^4$, training set size varies from 12 to 840 trajectories.
  • Figure 5.5: SA-NODEs, vanilla NODEs and exact solutions of transport equation \ref{['eq:trans_exam']} with initial measure \ref{['eq:trans_init_test']}.
  • ...and 3 more figures

Theorems & Definitions (25)

  • Theorem 2.1
  • Remark 2.2
  • Theorem 2.3
  • Remark 2.4
  • Remark 2.5: Comparison with Finite Element Approximation
  • Remark 2.6: Explicit formulation of constant
  • Theorem 2.7
  • Remark 2.8: Sharper Sobolev index
  • Remark 2.9: MPC perspective
  • Remark 2.10: Approximation in the $\mathbb{L}^p$-norm
  • ...and 15 more