Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guarantees

TL;DR

This work advances neural differential equations by establishing universal approximation guarantees over infinite time horizons for multistable dynamical systems. By exploiting Morse-Smale structural stability and exact period matching for limit cycles, it proves that Neural ODEs can approximate target flows with arbitrarily small trajectory error on $[0,\infty)$ except on a vanishing initial-condition set, while linking this topological precision to practical $L^p$ error metrics via a temporal generalization bound. It introduces the $\varepsilon$-$\delta$ closeness framework to separate precision from reliability and develops a tiling strategy for normally hyperbolic continuous attractors, enabling approximation by a finite discrete skeleton. The results provide the first universal-approximation framework for multistable infinite-horizon dynamics and offer a principled basis for diagnosing learning failures in neural dynamical systems along basin, phase, and discretization dimensions. Practical impact includes improved mechanistic interpretability and temporal fidelity of learned dynamics, though limitations such as learnability gaps and fragility of exact period locking motivate future work on architecture and training strategies for robust infinite-horizon modeling.

Abstract

Universal approximation theorems establish the expressive capacity of neural network architectures. For dynamical systems, existing results are limited to finite time horizons or systems with a globally stable equilibrium, leaving multistability and limit cycles unaddressed. We prove that Neural ODEs achieve $\varepsilon$-$δ$ closeness -- trajectories within error $\varepsilon$ except for initial conditions of measure $< δ$ -- over the \emph{infinite} time horizon $[0,\infty)$ for three target classes: (1) Morse-Smale systems (a structurally stable class) with hyperbolic fixed points, (2) Morse-Smale systems with hyperbolic limit cycles via exact period matching, and (3) systems with normally hyperbolic continuous attractors via discretization. We further establish a temporal generalization bound: $\varepsilon$-$δ$ closeness implies $L^p$ error $\leq \varepsilon^p + δ\cdot D^p$ for all $t \geq 0$, bridging topological guarantees to training metrics. These results provide the first universal approximation framework for multistable infinite-horizon dynamics.

Figures (1)

• Figure 1: The 1D bistable system (Example \ref{['ex:bistable']}). (a) Vector field $f(x) = x - x^3$ with stable $x_\pm = \pm 1$ (filled) and unstable $x_0 = 0$ (open); phase line shows basins with perturbed separatrix (dashed orange). (b) Trajectories from various initial conditions; orange region marks ICs that switch basins under perturbation $\hat{f} = f - \nu\cos(2x)$. (c) Time-averaged $L^2$ error vs. perturbation size $\nu$: empirical error (black) stays below bound $\varepsilon^2 + \delta D^2$ (dashed, Theorem \ref{['thm:lp_bound']}), decomposed into trajectory error (red) and B-type basin error (blue).

Theorems & Definitions (94)

• Remark : Direct vs. embedding approximation
• Definition 1: Hypothesis class with $C^1$ UAP
• Remark
• Definition 2: Neural ODE
• Definition 3: Hyperbolic fixed point
• Definition 4: Hyperbolic periodic orbit
• Definition 5: Morse-Smale system
• Theorem 1: Palis-Smale palis1970ss
• Remark : Morse-Smale systems
• Theorem 2: Stable Manifold Theorem chicone2006ode