RefreshNet: Learning Multiscale Dynamics through Hierarchical Refreshing

TL;DR

This work tackles the challenge of accurate long-horizon forecasting for multiscale dynamical systems, where error accumulation and computational cost hinder traditional autoregressive models. It introduces RefreshNet, a framework that learns a reduced latent space with a convolutional autoencoder and propagates dynamics through a hierarchy of LSTM-based RNN blocks operating at geometrically increasing timescales, complemented by a refreshing mechanism that resets inputs to finer blocks to curb error growth. The approach yields substantial improvements in long-term prediction accuracy and computational efficiency across three benchmark systems (FitzHugh-Nagumo, Reaction-Diffusion, and Kuramoto-Sivashinsky) and consistently outperforms state-of-the-art methods such as LED and LSTM. By enabling accurate, scalable forecasting of complex multiscale dynamics, RefreshNet offers a practical path toward efficient, data-driven modeling of large-scale systems in science and engineering, with potential extensions to generative modeling and exogenous-perturbation handling.

Abstract

Forecasting complex system dynamics, particularly for long-term predictions, is persistently hindered by error accumulation and computational burdens. This study presents RefreshNet, a multiscale framework developed to overcome these challenges, delivering an unprecedented balance between computational efficiency and predictive accuracy. RefreshNet incorporates convolutional autoencoders to identify a reduced order latent space capturing essential features of the dynamics, and strategically employs multiple recurrent neural network (RNN) blocks operating at varying temporal resolutions within the latent space, thus allowing the capture of latent dynamics at multiple temporal scales. The unique "refreshing" mechanism in RefreshNet allows coarser blocks to reset inputs of finer blocks, effectively controlling and alleviating error accumulation. This design demonstrates superiority over existing techniques regarding computational efficiency and predictive accuracy, especially in long-term forecasting. The framework is validated using three benchmark applications: the FitzHugh-Nagumo system, the Reaction-Diffusion equation, and Kuramoto-Sivashinsky dynamics. RefreshNet significantly outperforms state-of-the-art methods in long-term forecasting accuracy and speed, marking a significant advancement in modeling complex systems and opening new avenues in understanding and predicting their behavior.

Figures (17)

• Figure 1: Demonstration of iterative, autoregressive forecasting
• Figure 2: Classical Autoencoder
• Figure 3: Convolutional autoencoder
• Figure 4: Data sequences at different prediction steps in RefreshNet. Blue, red, and green colors represent the $\mathcal{R}_1$, $\mathcal{R}_2$, and $\mathcal{R}_3$ models and blue, red, and green fonts represents the data generated by them , respectively. Black fond represents the initial input data sequence. For clearer visualization, see Fig. \ref{['data_flow_example']} in the Appendix showing a case study for $k=1$.
• Figure 5: Illustration of RefreshNet architecture. The testing initial condition (IC) is passed through the encoder block to obtain the latent vector $z(t_i, \mu)$ which serves as the input to the different RNN blocks operating with different inference periods. The final predictions are transformed to the original space via the Decoder block.