New universal operator approximation theorem for encoder-decoder architectures (Preprint)
Janek Gödeke, Pascal Fernsel
TL;DR
The paper addresses universal operator approximation for encoder-decoder architectures implementing mappings G ∈ 𝒞(𝒳,𝒴) between infinite-dimensional spaces, focusing on uniform convergence on every compact subset of 𝒳. It adopts a topological viewpoint via the compact-open topology and introduces the encoder-decoder approximation property (EDAP), enabling K-independent universal approximation results. A central theorem shows that if both domain and range spaces have EDAP and 𝔽 is a universal function approximator, then there exists a sequence G_n = D_n^𝒴 ∘ φ_n ∘ E_n^𝒳 with φ_n ∈ 𝔽 that converges uniformly to any G on all compacts. The results unify classical DeepONets, MIONets, BasisONets, and frame-based encoders/decoders under one theoretical umbrella, extending prior work (e.g., Schwab2023) to a broader class of architectures and non-linear encoders/decoders, with implications for robust operator learning in PDEs and related fields.
Abstract
Motivated by the rapidly growing field of mathematics for operator approximation with neural networks, we present a novel universal operator approximation theorem for a broad class of encoder-decoder architectures. In this study, we focus on approximating continuous operators in $\mathcal{C}(\mathcal{X}, \mathcal{Y})$, where $\mathcal{X}$ and $\mathcal{Y}$ are infinite-dimensional normed or metric spaces, and we consider uniform convergence on compact subsets of $\mathcal{X}$. Unlike standard results in the operator learning literature, we investigate the case where the approximating operator sequence can be chosen independently of the compact sets. Taking a topological perspective, we analyze different types of operator approximation and show that compact-set-independent approximation is a strictly stronger property in most relevant operator learning frameworks. To establish our results, we introduce a new approximation property tailored to encoder-decoder architectures, which enables us to prove a universal operator approximation theorem ensuring uniform convergence on every compact subset. This result unifies and extends existing universal operator approximation theorems for various encoder-decoder architectures, including classical DeepONets, BasisONets, special cases of MIONets, architectures based on frames and other related approaches.