Beyond the Kolmogorov Barrier: A Learnable Weighted Hybrid Autoencoder for Model Order Reduction

TL;DR

This work introduces a Learnable Weighted Hybrid Autoencoder (LWH-AE) that blends POD/SVD with nonlinear neural encoders/decoders via learnable weights, enabling SVD-like convergence and improved generalization beyond the Kolmogorov barrier. The framework supports both pure reconstruction and time-dependent surrogate modeling by coupling with Koopman forecasting or LSTM dynamics, and demonstrates robust performance on chaotic and multi-scale PDEs including KS and HIT. Across KS, HIT, traveling wave, cylinder flow, Burgers, and shallow-water datasets, LWH-AE consistently outperforms POD, vanilla AE, and simple hybrid baselines, while achieving dramatically lower sharpness and greater noise robustness. The results indicate that high-quality reduced representations are the primary bottleneck in ROMs, and that the proposed approach yields practical gains for surrogate modeling and long-horizon predictions with minimal computational overhead.

Abstract

Representation learning for high-dimensional, complex physical systems aims to identify a low-dimensional intrinsic latent space, which is crucial for reduced-order modeling and modal analysis. To overcome the well-known Kolmogorov barrier, deep autoencoders (AEs) have been introduced in recent years, but they often suffer from poor convergence behavior as the rank of the latent space increases. To address this issue, we propose the learnable weighted hybrid autoencoder, a hybrid approach that combines the strengths of singular value decomposition (SVD) with deep autoencoders through a learnable weighted framework. We find that the introduction of learnable weighting parameters is essential -- without them, the resulting model would either collapse into a standard POD or fail to exhibit the desired convergence behavior. Interestingly, we empirically find that our trained model has a sharpness thousands of times smaller compared to other models. Our experiments on classical chaotic PDE systems, including the 1D Kuramoto-Sivashinsky and forced isotropic turbulence datasets, demonstrate that our approach significantly improves generalization performance compared to several competing methods. Additionally, when combining with time series modeling techniques (e.g., Koopman operator, LSTM), the proposed technique offers significant improvements for surrogate modeling of high-dimensional multi-scale PDE systems.

Figures (20)

• Figure 1: Architecture of the simple hybrid autoencoder and learnable weighted hybrid autoencoder.
• Figure 2: Generalization performance of four models on 1D Kuramoto-Shivaskinsky dataset with varying rank $r$ and resolution $N$. X axis denotes the rank of the latent space and Y axis denotes the Mean Testing $L^2$ error. For each grid, rank, and method, the model is trained across 16 independent runs with varying random seed values. The results from these runs are averaged to compute the mean testing $L^2$ error. The convergence rate for POD and Learnable weighted hybrid approach is indicated in the legend as an exponent to $r$.
• Figure 3: Generalization performance of four models on 3D homogeneous isotropic turbulence dataset with varying rank $r$ and resolution $N$. The testing $L^2$ error obtained using ReLU activation function is indicated using dashed lines. Similar to K-S case, 16 independent runs with varying random seed values are performed to obtain the mean testing $L^2$ error.
• Figure 4: Generalization performance comparison on 3D HIT data at a resolution of $64^3$ and rank $r=5$, with a 2D slice of $u_y$. Top: $u_y$. Bottom: absolute error of $u_y$ between the reconstructed field and ground truth.
• Figure 5: Generalization performance of the Koopman decoder model on traveling wave and flow over cylinder dataset. Latent space rank of 2 and 4 was used for the traveling wave cylinder case respectively. Different methods for dimensionality reduction are represented by different colors. Y axis represents the Mean Testing $L^2$ error.