WENDy for Nonlinear-in-Parameters ODEs
Nic Rummel, Daniel A. Messenger, Stephen Becker, Vanja Dukic, David M. Bortz
TL;DR
This work extends weak-form learning (WENDy) to nonlinear-in-parameters ODEs by deriving a maximum likelihood estimator (WENDy-MLE) that leverages analytic first- and second-order derivatives of the weak residual. It models data noise via additive Gaussian and multiplicative log-normal distributions, derives the weak residual distribution and a closed-form negative log-likelihood ell(p) = $\log\det\mathbf{S}(\mathbf{p}) + \mathbf{r}(\mathbf{p})^{\top}\mathbf{S}(\mathbf{p})^{-1}\mathbf{r}(\mathbf{p})$, and extends to log-normal noise with a transformed state. The authors implement a Julia package and demonstrate through extensive benchmarks that WENDy-MLE achieves higher accuracy, a larger domain of convergence, and competitive or faster performance than OE-LS and prior weak-form methods, even under challenging noise and nonlinearities. The approach provides reliable parameter uncertainty through an approximate estimator distribution and yields practical robustness without forward-solve-based optimization, enabling scalable identification for complex ODE systems. The work discusses robustness to noise, identified biases from nonlinearity, and computational costs, outlining directions for further improvement and extension to larger dynamical systems and PDEs.
Abstract
The Weak-form Estimation of Non-linear Dynamics (WENDy) framework is a recently developed approach for parameter estimation and inference of systems of ordinary differential equations (ODEs). Prior work demonstrated WENDy to be robust, computationally efficient, and accurate, but only works for ODEs which are linear-in-parameters. In this work, we derive a novel extension to accommodate systems of a more general class of ODEs that are nonlinear-in-parameters. Our new WENDy-MLE algorithm approximates a maximum likelihood estimator via local non-convex optimization methods. This is made possible by the availability of analytic expressions for the likelihood function and its first and second order derivatives. WENDy-MLE has better accuracy, a substantially larger domain of convergence, and is often faster than other weak form methods and the conventional output error least squares method. Moreover, we extend the framework to accommodate data corrupted by multiplicative log-normal noise. The WENDy.jl algorithm is efficiently implemented in Julia. In order to demonstrate the practical benefits of our approach, we present extensive numerical results comparing our method, other weak form methods, and output error least squares on a suite of benchmark systems of ODEs in terms of accuracy, precision, bias, and coverage.