A practical existence theorem for reduced order models based on convolutional autoencoders
Nicola Rares Franco, Simone Brugiapaglia
TL;DR
The work provides a practical existence theorem for CNN-based autoencoder DL-ROMs by embedding them in a holomorphic-regularity framework and leveraging sparse high-dimensional approximation. It proves that there exist encoder/decoder networks and a reduced map with provable error bounds that separate a sampling term (exponentially decaying with training size) from an approximation term tied to latent dimension and smoothness. The theory formalizes practical training choices, such as latent regularization and convolutional decoders, and demonstrates applicability to a parametric diffusion problem in 1D, yielding explicit error bounds and architectural guidelines. While the results offer substantial theoretical grounding and align with practitioner heuristics, they highlight limitations in higher dimensions and the need for further research on extending the framework and improving scalability.
Abstract
In recent years, deep learning has gained increasing popularity in the fields of Partial Differential Equations (PDEs) and Reduced Order Modeling (ROM), providing domain practitioners with new powerful data-driven techniques such as Physics-Informed Neural Networks (PINNs), Neural Operators, Deep Operator Networks (DeepONets) and Deep-Learning based ROMs (DL-ROMs). In this context, deep autoencoders based on Convolutional Neural Networks (CNNs) have proven extremely effective, outperforming established techniques, such as the reduced basis method, when dealing with complex nonlinear problems. However, despite the empirical success of CNN-based autoencoders, there are only a few theoretical results supporting these architectures, usually stated in the form of universal approximation theorems. In particular, although the existing literature provides users with guidelines for designing convolutional autoencoders, the subsequent challenge of learning the latent features has been barely investigated. Furthermore, many practical questions remain unanswered, e.g., the number of snapshots needed for convergence or the neural network training strategy. In this work, using recent techniques from sparse high-dimensional function approximation, we fill some of these gaps by providing a new practical existence theorem for CNN-based autoencoders when the parameter-to-solution map is holomorphic. This regularity assumption arises in many relevant classes of parametric PDEs, such as the parametric diffusion equation, for which we discuss an explicit application of our general theory.