Learning functional components of PDEs from data using neural networks

TL;DR

This work embeds neural networks into the PDE and shows how, as they are trained on data, they can approximate unknown functions with arbitrary accuracy and be treated as a normal PDE for purposes such as generating system predictions.

Abstract

Partial differential equations often contain unknown functions that are difficult or impossible to measure directly, hampering our ability to derive predictions from the model. Workflows for recovering scalar PDE parameters from data are well studied: here we show how similar workflows can be used to recover functions from data. Specifically, we embed neural networks into the PDE and show how, as they are trained on data, they can approximate unknown functions with arbitrary accuracy. Using nonlocal aggregation-diffusion equations as a case study, we recover interaction kernels and external potentials from steady state data. Specifically, we investigate how a wide range of factors, such as the number of available solutions, their properties, sampling density, and measurement noise, affect our ability to successfully recover functions. Our approach is advantageous because it can utilise standard parameter-fitting workflows, and in that the trained PDE can be treated as a normal PDE for purposes such as generating system predictions.

Figures (25)

• Figure 1: Diagram of standard workflow for analysing PDE recovery from data. (1) In the first step, we determine the functional forms for $W$ and $V$, and the value of $\kappa$, for the PDE we wish to analyse. Full list of values used for all PDEs analysed in this work can be found in Table \ref{['tab:PDE_list']}. (2.1) From the PDE, we compute its true steady state solution profiles using the approach described in carrillosalmaniwvillares2025. (2.2) In Section \ref{['sec:results_noisy_fits']}, we use the approach described in Supplementary Section \ref{['sup_section:detailed_methods_noisy_data']} to generate downsampled, noisy, solution profiles. (3) First, we assume that some subset of the parameters is unknown (in Sections \ref{['sec:results_normal_fits']} and \ref{['sec:results_noisy_fits']} only $W$ is assumed to be unknown, while Sections \ref{['sec:results_WVk_fits']} consider multiple unknown parameters). Next, we attempt to recover these from measured solution profiles using the approach described in Section \ref{['section:methods_udes']}. The fitting process can be evaluated by comparing the fitted parameters to the true ones (as decided in 1). (4) Finally, the fitting process can be further evaluated by computing the steady state solutions for the fitted PDE, which then can be compared to true ones computed in 2.1.
• Figure 1: Functional parameters can be recovered using an alternative loss function. To show that our approach is not restricted to the loss function presented in Section \ref{['sec:identification_of_loss']} (Equation \ref{['eq:loss_function_FP']}), we replicate them using another loss function presented in the same section (Equation \ref{['eq:loss_function_PDE']}). (A-D) For the four different parameterisations of the PDE (each also used in Supplementary Figure \ref{['fig:fig_1_sup_alt_Ws']}), we carry out the procedure described in Figure \ref{['fig:workflow_flowchart']}. In each case, using the loss function in Equation \ref{['eq:loss_function_PDE']}, the interaction kernel is correctly recovered.
• Figure 2: The interaction kernel $W$ can be recovered from solution profiles. (A) For a nonlocal PDE (Equation \ref{['eq:model']}) we set the potential $V \equiv 0$, the interaction strength parameter $\kappa = 8$, and use a multi-modal interaction kernel $W$ (described in Supplementary Section \ref{['sup_section:pde_list']}, depicted with solid blue line). (B) From the fully-determined PDE, we compute the set of four potential solutions (solid green lines). (C) Assuming $W$ is unknown, it can be recovered by fitting a neural network to the solution profiles (Section \ref{['section:methods_udes']}). We confirm that the fitted $W^*$ (dashed red line) replicates the true $W$ (solid blue line). (D) Finally, we compute the solutions for the PDE instance $(W^*,V,\kappa)$ (dashed purple lines), confirming these correspond to the ground truth solutions (solid green lines).
• Figure 2: A Fourier mode expansion can be used as a universal function approximator. Throughout this paper, we rely on neural networks to approximate the functional parameters $V$ and $W$. However, other approaches are possible. Here, for four different $W$ (in all cases, $V\equiv0$), we perform the workflow described in Figure \ref{['fig:workflow_flowchart']} to recover $W$ from the solution profiles. Instead of using neural networks to approximate the unknown functions, we use Fourier mode expansion (detailed in Supplementary Section \ref{['sup_section:fouier_expansion_ufa']}). (A-D) In all four cases, the Fourier mode expansion can successfully recover $W$ from the steady state solutions, and the fitted PDE yields the ground-truth solution profiles.
• Figure 3: The interaction kernel $W$ can be recovered from noisy steady state solutions. (A) The solutions of the PDE described in Figure \ref{['fig:fit_W_to_sols']}. (B-D) We downsample the solution in A to $100$ datapoints, and also add low (B), medium (C), and high (D) levels of noise (Supplementary Section \ref{['sup_section:detailed_methods_noisy_data']}). (E-G) Using the optimisation procedure from Figure \ref{['fig:workflow_flowchart']}, we attempt to recover $W$ using the noisy solution profiles. For low (E) and medium (F) noise levels, the fitted $W^*$ (dashed red lines) follows the ground truth $W$ (solid blue lines). However, for the high noise levels (G), we can no longer accurately recover $W$.

Theorems & Definitions (3)

• Proposition A.1
• Proposition A.2
• Proposition A.3