Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning

TL;DR

The paper introduces the Fourier Learning Machine (FLM), a neural network architecture that learns a multidimensional nonharmonic Fourier series by training frequencies, amplitudes, and phase shifts using cosine activations within a compact, separable basis. It achieves a complete, interpretable Fourier basis in $m$ dimensions via the cosine phase–shifted representation and the $m$-Lexi Sign Matrix, enabling spectral adaptability to both periodic and nonperiodic functions in a standard MLP framework. Empirical results on benchmark PDEs and optimal control problems show FLM's performance is competitive with, and often superior to, established architectures like SIREN and vanilla NNs, while offering greater interpretability. The work promises practical impact for scientific machine learning by bridging classical spectral methods with data-driven neural models for high-dimensional surrogate modeling and analysis.

Abstract

We introduce the Fourier Learning Machine (FLM), a neural network (NN) architecture designed to represent a multidimensional nonharmonic Fourier series. The FLM uses a simple feedforward structure with cosine activation functions to learn the frequencies, amplitudes, and phase shifts of the series as trainable parameters. This design allows the model to create a problem-specific spectral basis adaptable to both periodic and nonperiodic functions. Unlike previous Fourier-inspired NN models, the FLM is the first architecture able to represent a multidimensional Fourier series with a complete set of basis functions in separable form, doing so by using a standard Multilayer Perceptron-like architecture. A one-to-one correspondence between the Fourier coefficients and amplitudes and phase-shifts is demonstrated, allowing for the translation between a full, separable basis form and the cosine phase-shifted one. Additionally, we evaluate the performance of FLMs on several scientific computing problems, including benchmark Partial Differential Equations (PDEs) and a family of Optimal Control Problems (OCPs). Computational experiments show that the performance of FLMs is comparable, and often superior, to that of established architectures like SIREN and vanilla feedforward NNs.

Figures (9)

• Figure 1: A "clump" of neurons that outputs the $i^{th}$ single $cosine$ basis of the Fourier Network appearing in GW88.
• Figure 2: FLM with two inputs. The sub--network associated with frequency pair $\boldsymbol{n}$ (Left) and the main network with $N$ sub--networks (Right).
• Figure 3: $(a)$ FLM with 3 inputs and the $S^{(3)}$ matrix defining the sign of the frequency components $n_1$, $n_2$ and $n_3$. These components are illustrated, respectively, in $(b)-(d)$.
• Figure 4: $m-$dimensional FLM sub--network (biases are not shown).
• Figure 5: Solution surfaces of the Heat (a), Poisson (b), GBS (c), and Burgers' (d) equations, showing exact solution (left), FLM approximation (middle), and absolute error (right).

Theorems & Definitions (9)

• Definition 1: $m$--Lexi Sign Matrix
• Definition 2: Single-Layer Feedforward Neural Network
• Definition 3: Fourier Learning Machine (FLM)
• Example 4.1: FLM Sub--networks with Two Inputs
• Example 4.2: FLM Sub--Networks with $m$ inputs
• Example 5.1: One--Dimensional Heat Equation
• Example 5.2: Two-Dimensional Poisson Equation
• Example 5.3: Generalized Black-Scholes Equation
• Example 5.4: Inviscid Burgers' Equation