Quantitative Flow Approximation Properties of Narrow Neural ODEs
Karthik Elamvazhuthi
TL;DR
The paper investigates how narrow neural ODEs, whose parameter count scales with the input dimension, can approximate the flows of wider NODEs and hence a broad class of dynamical systems. It introduces a simple, Grönwall-based proof strategy that avoids heavy control-theoretic machinery by analyzing the switching between a finite set of vector fields $f_i(x)=A_i\Sigma(W_i x+b_i)$ to emulate a composite velocity field. The main result provides a explicit error bound, showing how the approximation error decays with the number of switches $N$ and depends on constants derived from Lipschitz continuity and linear-growth bounds. This establishes quantitative rates for uniform-flow approximation by narrow NODEs and supports their ability to replicate flows modeled by shallow but wide networks.
Abstract
In this note, we revisit the problem of flow approximation properties of neural ordinary differential equations (NODEs). The approximation properties have been considered as a flow controllability problem in recent literature. The neural ODE is considered {\it narrow} when the parameters have dimension equal to the input of the neural network, and hence have limited width. We derive the relation of narrow NODEs in approximating flows of shallow but wide NODEs. Due to existing results on approximation properties of shallow neural networks, this facilitates understanding which kind of flows of dynamical systems can be approximated using narrow neural ODEs. While approximation properties of narrow NODEs have been established in literature, the proofs often involve extensive constructions or require invoking deep controllability theorems from control theory. In this paper, we provide a simpler proof technique that involves only ideas from ODEs and Gr{ö}nwall's lemma. Moreover, we provide an estimate on the number of switches needed for the time dependent weights of the narrow NODE to mimic the behavior of a NODE with a single layer wide neural network as the velocity field.