Investigating the Surrogate Modeling Capabilities of Continuous Time Echo State Networks
Saakaar Bhatnagar
TL;DR
This work investigates CTESNs as surrogate models for stiff ODEs, focusing on how the reservoir-to-output projection affects generalization. It systematically compares linear and nonlinear projections, showing that NLPCTESN outperforms LPCTESN across benchmark problems (Robertson, Sliding Basepoint, POLLU) and across parameterizations of rates and initial conditions, with further gains from polynomial-augmented k-NN RBF interpolation. The CTESN surrogates deliver accurate predictions over challenging timescales and offer speedups of up to two orders of magnitude over traditional ODE solvers, enabling real-time prediction and potential digital-twin applications in engineering systems. The study provides practical guidance on algorithmic choices and hyper-parameter settings for CTESNs in stiff dynamical systems and motivates future physics-informed extensions for broader, high-dimensional problems.
Abstract
Continuous Time Echo State Networks (CTESNs) are a promising yet under-explored surrogate modeling technique for dynamical systems, particularly those governed by stiff Ordinary Differential Equations (ODEs). A key determinant of the generalization accuracy of a CTESN surrogate is the method of projecting the reservoir state to the output. This paper shows that of the two common projection methods (linear and nonlinear), the surrogates developed via the nonlinear projection consistently outperform those developed via the linear method. CTESN surrogates are developed for several challenging benchmark cases governed by stiff ODEs, and for each case, the performance of the linear and nonlinear projections is compared. The results of this paper demonstrate the applicability of CTESNs to a variety of problems while serving as a reference for important algorithmic and hyper-parameter choices for CTESNs