Factorized Fourier Neural Operators

TL;DR

The paper tackles the challenge of efficiently and accurately simulating PDEs across diverse geometries using neural operators. It introduces the Factorized Fourier Neural Operator (F-FNO), which employs separable Fourier factorization, residual-after-activation layers, and a two-layer feedforward, together with training strategies like teacher forcing and cosine learning rate decay to enable deep networks with far fewer parameters. Empirical results show substantial error reductions over prior FNO and geo-FNO methods and significant speedups compared with pseudo-spectral solvers on Navier–Stokes, elasticity, and aero/elasticity problems, including irregular geometries and 3D outputs. The work highlights flexible input representations and strong scalability, while also pointing to theoretical questions about infinite-depth behavior and the universal approximation properties under Fourier factorization. Overall, F-FNO provides a powerful, adaptable framework for fast, accurate PDE surrogates across a range of complex geometries and dynamics.

Abstract

We propose the Factorized Fourier Neural Operator (F-FNO), a learning-based approach for simulating partial differential equations (PDEs). Starting from a recently proposed Fourier representation of flow fields, the F-FNO bridges the performance gap between pure machine learning approaches to that of the best numerical or hybrid solvers. This is achieved with new representations - separable spectral layers and improved residual connections - and a combination of training strategies such as the Markov assumption, Gaussian noise, and cosine learning rate decay. On several challenging benchmark PDEs on regular grids, structured meshes, and point clouds, the F-FNO can scale to deeper networks and outperform both the FNO and the geo-FNO, reducing the error by 83% on the Navier-Stokes problem, 31% on the elasticity problem, 57% on the airfoil flow problem, and 60% on the plastic forging problem. Compared to the state-of-the-art pseudo-spectral method, the F-FNO can take a step size that is an order of magnitude larger in time and achieve an order of magnitude speedup to produce the same solution quality.

Figures (9)

• Figure 1: An illustration of the input and output of different PDE problems. See the accompanying \ref{['tab:datasets']} for details. On the torus datasets (a), the operator learns to evolve the vorticity over time. On Elasticity (b), the operator learns to predict the stress value on each point on a point cloud. On Airfoil (c), the operator learns to predict the flow velocity on each mesh point. On Plasticity (d), the operator learns the displacement of each mesh point given an initial boundary condition.
• Figure 2: The architecture of the F-FNO for a 2D problem. The iterative process (\ref{['eqn:process']}) is shown at the top, in which the input function $a(i,j)$ is first deformed from an irregular space into a uniform space $a(x,y)$, and is then fed through a series of operator layers $\mathcal{L}$ in order to produce the output function $u(i,j)$. A zoomed-in operator layer (\ref{['eqn:layer']}) is shown at the bottom which shows how we process each spatial dimension independently in the Fourier space, before merging them together again in the physical space.
• Figure 3:
• Figure 6:
• Figure 9: Performance of F-FNO on different contexts and input representations.