Hierarchical-embedding autoencoder with a predictor (HEAP) as efficient architecture for learning long-term evolution of complex multi-scale physical systems

TL;DR

HEAP introduces a Hierarchical-Embedding Autoencoder-Predictor that encodes system states into multiple scale-specific embeddings and predicts forward in embedding space with a synchronized, cross-scale predictor. The hierarchical autoencoder preserves spatial information across scales, while the predictor enables interactions within and between layers, reducing computational cost and data requirements. In a 2D Hasegawa–Wakatani turbulence case, HEAP with 3 embedding levels (H3) substantially improves long-rollout statistics over conventional FCAE+ResNet baselines, evidenced by spectral and autocorrelation metrics, including the $E(k)$ spectra. The approach offers a data-efficient, scalable surrogate for multi-scale physics, with potential applications as sub-grid modeling or reduced-order representations in turbulence and related systems.

Abstract

We propose a novel efficient architecture for learning long-term evolution in complex multi-scale physical systems which is based on the idea of separation of scales. Structures of various scales that dynamically emerge in the system interact with each other only locally. Structures of similar scale can interact directly when they are in contact and indirectly when they are parts of larger structures that interact directly. This enables modeling a multi-scale system in an efficient way, where interactions between small-scale features that are apart from each other do not need to be modeled. The hierarchical fully-convolutional autoencoder transforms the state of a physical system not just into a single embedding layer, as it is done conventionally, but into a series of embedding layers which encode structures of various scales preserving spatial information at a corresponding resolution level. Shallower layers embed smaller structures on a finer grid, while deeper layers embed larger structures on a coarser grid. The predictor advances all embedding layers in sync. Interactions between features of various scales are modeled using a combination of convolutional operators. We compare the performance of our model to variations of a conventional ResNet architecture in application to the Hasegawa-Wakatani turbulence. A multifold improvement in long-term prediction accuracy was observed for crucial statistical characteristics of this system.

Figures (14)

• Figure 1: Various architectures of convolutional autoencoders. Blue rectangles represent encoded data, in a single layer (a) and (b) and in multiple hierarchical layers (c). Vertical rectangles represent 2D data (on a 2D grid); a horizontal rectangle represents 1D (flattened) data. The width of vertical rectangles indicates the number of channels. Arrows represent the flow of information.
• Figure 2: Multi-scale nature of structures encoded by various encoder layers. Deeper layers encode larger structures.
• Figure 3: Predictors to couple with a conventional fully-convolutional AE (a) and hierarchical AE (b)
• Figure 4: Variants of the models tested; c denotes of the number of embedding channels. Each trapezoid denotes a single convolutional layer with a 4x4 filter and stride 2.
• Figure 5: Comparing spatial spectra of recovered $n$ and $\phi$ fields with various autoencoders (a). Normalized overall deviation from the ground truth (b), and training time (c).