Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Ritz method with Fourier feature mapping: A deep learning approach for solving variational models of microstructure

Ensela Mema, Ting Wang, Jaroslaw Knap

TL;DR

The results demonstrate that Fourier feature mapping enables DRM to generate high-frequency, multiscale solutions for the benchmark problems in both 1D and 2D, offering a promising advancement in tackling complex non-convex energy minimization problems.

Abstract

This paper presents a novel approach that combines the Deep Ritz Method (DRM) with Fourier feature mapping to solve minimization problems comprised of multi-well, non-convex energy potentials. These problems present computational challenges as they lack a global minimum. Through an investigation of three benchmark problems in both 1D and 2D, we observe that DRM suffers from spectral bias pathology, limiting its ability to learn solutions with high frequencies. To overcome this limitation, we modify the method by introducing Fourier feature mapping. This modification involves applying a Fourier mapping to the input layer before it passes through the hidden and output layers. Our results demonstrate that Fourier feature mapping enables DRM to generate high-frequency, multiscale solutions for the benchmark problems in both 1D and 2D, offering a promising advancement in tackling complex non-convex energy minimization problems.

Deep Ritz method with Fourier feature mapping: A deep learning approach for solving variational models of microstructure

TL;DR

The results demonstrate that Fourier feature mapping enables DRM to generate high-frequency, multiscale solutions for the benchmark problems in both 1D and 2D, offering a promising advancement in tackling complex non-convex energy minimization problems.

Abstract

This paper presents a novel approach that combines the Deep Ritz Method (DRM) with Fourier feature mapping to solve minimization problems comprised of multi-well, non-convex energy potentials. These problems present computational challenges as they lack a global minimum. Through an investigation of three benchmark problems in both 1D and 2D, we observe that DRM suffers from spectral bias pathology, limiting its ability to learn solutions with high frequencies. To overcome this limitation, we modify the method by introducing Fourier feature mapping. This modification involves applying a Fourier mapping to the input layer before it passes through the hidden and output layers. Our results demonstrate that Fourier feature mapping enables DRM to generate high-frequency, multiscale solutions for the benchmark problems in both 1D and 2D, offering a promising advancement in tackling complex non-convex energy minimization problems.

Paper Structure

This paper contains 14 sections, 1 theorem, 54 equations, 10 figures.

Table of Contents

  1. Introduction
  2. Deep Ritz Algorithm
  3. NTK analysis for Deep-Ritz and the Fourier feature
  4. The spectral bias pathology for DRM
  5. Fourier feature from the NTK perspective
  6. Numerical Results & Discussion
  7. $1D$ Benchmark Problem # 1
  8. $1D$ Benchmark Problem # 2
  9. $2D$ Benchmark Problem
  10. Regularized 2D Problem & Fourier Mapping
  11. Conclusions
  12. Acknowledgment
  13. Derivation of the gradient dynamics
  14. The eigenspectrum of the Gram matrix

Key Result

Theorem 1

Suppose that Then, the asymptotic gradient (with respect to $u$ and $u^{\prime}$) dynamics of the Lagrangian $W$ in DRM is given by eqn:gradient-evolution. Moreover, we have where we have used the spectral decomposition $D_{\mathcal{X}}M_{\mathcal{X}} = Q \Lambda Q^{\top}$ with orthonormal matrix $Q = [q_1, \ldots, q_{2|\mathcal{X}|}]$ and diagonal matrix $\Lambda = \text{diag}(\lambda_1, \ldots

Figures (10)

  • Figure 1: Structure of Neural Network in Deep Ritz Method.
  • Figure 2: Structure of Neural Network by applying Fourier feature mapping to the input layer.
  • Figure 3: (a) DRM approximation to \ref{['eqn:1D_Problem_1']} with ReLU activation function, $\eta = 1.0\times 10^{-4}$ after $100000$ epochs with DNN structure of $5$, $7$ and $9$ hidden layers. (b) The derivative $u_x$ of the DRM approximation to \ref{['eqn:1D_Problem_1']}.
  • Figure 4: DRM approximation to \ref{['eqn:1D_Problem_1']} where a NN with $5$,$7$ and $9$-hidden layers, ReLU activation function, $\eta = 1.0\times 10^{-4}$ and Fourier mapping of frequency $\delta({\bf x}) = \left[\sin(2^i\pi {\bf x}),\cos(2^i\pi {\bf x})\right]$ after $100000$ epochs.
  • Figure 5: First Row (a)-(c): DRM approximation to \ref{['eqn:1D_Problem_2']} with ReLU activation function, $\varepsilon = 0$, $\eta = 1.0\times 10^{-4}$ and cosine annealing after $200000$ epochs. Second Row (d)-(f): Row: DRM approximation to \ref{['eqn:1D_Problem_2']} with ReLU activation function, $\varepsilon = 0$, $\eta = 1.0\times 10^{-4}$ and cosine annealing after $500000$ epochs.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Theorem 1