Frequency-adaptive Multi-scale Deep Neural Networks

TL;DR

The paper tackles the difficulty of learning high-frequency and multi-scale functions with deep nets, proposing frequency-adaptive MscaleDNNs that jointly leverage down-scaling and Fourier-feature insights. It establishes a theoretical error bound for MscaleDNNs and Fourier-feature nets, then introduces a hybrid feature embedding and a posterior error estimate to drive an adaptive training loop that identifies and exploits dominant frequencies. The resulting framework yields two to three orders-of-magnitude improvements over standard MscaleDNNs across Poisson, Heat, Wave, and Schrödinger-type PDEs near the semi-classical limit. This approach enhances robustness to unknown frequency content and offers a practical pathway for solving complex multi-scale PDEs with neural networks.

Abstract

Multi-scale deep neural networks (MscaleDNNs) with downing-scaling mapping have demonstrated superiority over traditional DNNs in approximating target functions characterized by high frequency features. However, the performance of MscaleDNNs heavily depends on the parameters in the downing-scaling mapping, which limits their broader application. In this work, we establish a fitting error bound to explain why MscaleDNNs are advantageous for approximating high frequency functions. Building on this insight, we construct a hybrid feature embedding to enhance the accuracy and robustness of the downing-scaling mapping. To reduce the dependency of MscaleDNNs on parameters in the downing-scaling mapping, we propose frequency-adaptive MscaleDNNs, which adaptively adjust these parameters based on a posterior error estimate that captures the frequency information of the fitted functions. Numerical examples, including wave propagation and the propagation of a localized solution of the schr$\ddot{\text{o}}$dinger equation with a smooth potential near the semi-classical limit, are presented. These examples demonstrate that the frequency-adaptive MscaleDNNs improve accuracy by two to three orders of magnitude compared to standard MscaleDNNs.

Figures (13)

• Figure 2.1: A structure of MscaleDNNs.
• Figure 4.1: The relative $L_2$ error at each training epoch: standard DNNs (dotted blue line), MscaleDNNs with $k=38\pi$ (dashed orange line), and MscaleDNNs with $k=40\pi$ (solid green line), respectively.
• Figure 4.2: $k=40\pi$
• Figure 4.3: $k=38\pi$
• Figure 4.5: The relative $L_2$ errors after 100,000 training epochs: MscaleDNNs (dotted blue line), DNNs with Fourier embedding (dashed orange line), and DNNs with newly proposed hybrid feature embedding (solid green line) for $k\in [35\pi, 45\pi]$, respectively.

Theorems & Definitions (13)

• Definition 3.1
• Definition 3.2
• Definition 3.3
• Definition 3.4
• Theorem 3.1
• Remark 3.1
• Theorem 3.2
• proof
• Theorem 3.3
• proof