Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximations

TL;DR

Balanced Neural ODEs (B-NODEs) fuse variational autoencoders with Neural ODEs to create fast, probabilistic surrogate models that adaptively balance dimensionality reduction with reconstruction accuracy under time-varying inputs. By continuously propagating variational parameters through time, B-NODEs enable flexible latent dynamics and automatic model-order reduction without predefining latent dimensionality, while also supporting Koopman operator approximations. The method demonstrates substantial MOR gains and substantial speedups across academic and real-world use cases, including a discretized heat-flow system and a thermal power plant surrogate, with demonstrated potential for uncertainty quantification. Reproducibility is supported by released code and FMUs, and the framework is positioned to benefit real-time optimization, control, and large-scale simulation tasks in engineering domains.

Abstract

Variational Autoencoders (VAEs) are a powerful framework for learning latent representations of reduced dimensionality, while Neural ODEs excel in learning transient system dynamics. This work combines the strengths of both to generate fast surrogate models with adjustable complexity reacting on time-varying inputs signals. By leveraging the VAE's dimensionality reduction using a nonhierarchical prior, our method adaptively assigns stochastic noise, naturally complementing known NeuralODE training enhancements and enabling probabilistic time series modeling. We show that standard Latent ODEs struggle with dimensionality reduction in systems with time-varying inputs. Our approach mitigates this by continuously propagating variational parameters through time, establishing fixed information channels in latent space. This results in a flexible and robust method that can learn different system complexities, e.g. deep neural networks or linear matrices. Hereby, it enables efficient approximation of the Koopman operator without the need for predefining its dimensionality. As our method balances dimensionality reduction and reconstruction accuracy, we call it Balanced Neural ODE (B-NODE). We demonstrate the effectiveness of this methods on several academic and real-world test cases, e.g. a power plant or MuJoCo data.

Figures (18)

• Figure 1: Insights in the information transmission through the latent space. \ref{['fig:dim_read_heatmap']} is the result of a $\beta$-VAE ($\beta=1.0e-3$) with a latent space of $n_{\boldsymbol{z}}=512$ trained on a dataset of a thermal network model, generated by random sampling of 14 simulation model parameters (details in \ref{['app:VAE_dim_red_experimental_details']}).
• Figure 2: Scheme of the Balanced Neural ODE.
• Figure 3: Comparison of state reduction of B-NODE and vanilla Latent ODE RubanovaChen2019 for discretized heat flow example.
• Figure 4: Generalization and comparison of methods for discretized heat flow example.
• Figure 5: Error comparison key quantities of interest (outputs) for surrogates of power plant.