Tracking Finite-Time Lyapunov Exponents to Robustify Neural ODEs
Tobias Wöhrer, Christian Kuehn
TL;DR
This work introduces finite-time Lyapunov exponents (FTLEs) as dynamical systems tools to analyze neural ODEs and reveal how input perturbations propagate along the learned flow. By linking FTLE ridges to decision boundaries through Lagrangian coherent structures, the authors provide an interpretable view of how two-dimensional neural ODE classifiers separate classes and how early dynamics shape classification. They then propose an FTLE-based regularizer that suppresses large exponents in the early phase of the input trajectory, improving adversarial robustness while reducing compute compared to full-interval regularization. Experiments on two-dimensional moons-like tasks show that FTLE ridges coincide with class boundaries, and early-dynamics suppression widens margins without sacrificing accuracy, highlighting a practical dynamical-systems approach to robustness in continuous-depth models. The work offers a principled framework for interpreting and stabilizing neural ODEs and points toward scalable extensions to higher dimensions.
Abstract
We investigate finite-time Lyapunov exponents (FTLEs), a measure for exponential separation of input perturbations, of deep neural networks within the framework of continuous-depth neural ODEs. We demonstrate that FTLEs are powerful organizers for input-output dynamics, allowing for better interpretability and the comparison of distinct model architectures. We establish a direct connection between Lyapunov exponents and adversarial vulnerability, and propose a novel training algorithm that improves robustness by FTLE regularization. The key idea is to suppress exponents far from zero in the early stage of the input dynamics. This approach enhances robustness and reduces computational cost compared to full-interval regularization, as it avoids a full ``double'' backpropagation.