The Neural Network Approach to Inverse Problems in Differential Equations

TL;DR

This work develops a neural-network-based framework to solve inverse problems in differential equations by learning unknown functions inside PDEs with automatic differentiation and deferred discretization. It performs a rigorous error analysis that decomposes total error into observation noise, discretization (consistency), and optimization components, deriving a bound for diffusion problems that links |f_theta(x) - f(x)| to Δt, h, ε_opt, and ε_o. The methodology enables end-to-end calibration by evaluating neural networks within forward solvers and using a data-driven loss, while also providing a sensitivity analysis to quantify how small changes in network parameters affect quantities of interest. Numerical experiments on diffusion, wave, and Burgers-type equations demonstrate second-order convergence with data refinement, robustness to noise, and the ability to recover complex, spatially varying coefficients. The framework offers a scalable, practical approach to inverse PDE problems with potential extensions to tailored AD tools, advanced architectures, and uncertainty quantification through sensitivity metrics.

Abstract

We proposed a framework for solving inverse problems in differential equations based on neural networks and automatic differentiation. Neural networks are used to approximate hidden fields. We analyze the source of errors in the framework and derive an error estimate for a model diffusion equation problem. Besides, we propose a way for sensitivity analysis, utilizing the automatic differentiation mechanism embedded in the framework. It frees people from the tedious and error-prone process of deriving the gradients. Numerical examples exhibit consistency with the convergence analysis and error saturation is noteworthily predicted. We also demonstrate the unique benefits neural networks offer at the same time: universal approximation ability, regularizing the solution, bypassing the curse of dimensionality and leveraging efficient computing frameworks.

Figures (9)

• Figure 1: The width of the stripe describes the impact of a small change in the neural network parameters if the maximum value of the fitted value is the targeted physical quantity.
• Figure 2: The snapshots are taken at $t=0.1$ and $t=t+\Delta t$, with $\Delta t=0.001$ and $h=0.002$. We can see that the calibrated value of $c(x)$ matches perfectly with the exact value.
• Figure 3: We take the snapshots at $t=0.1$ and $t=t+\Delta t$, with various $\Delta t$ and $h$, and $n = \frac{2}{h}$ in the plots. We see a clear pattern exactly predicted in \ref{['thm:main']}: a second order convergence with respect to $\Delta t$ and $h$, and at some point, the error ceases to decrease and remains at a constant level.
• Figure 4: Sensitivity analysis for \ref{['equ:ex1']}. The physical quantity we are interested is the conductivity function value at $x=0$, i.e., the prediction for $c(0)$. We have used three different step size $\alpha=0.001$, $0.002$ and $0.003$. The blue region is the sensitivity region defined by \ref{['equ:sensitivity']}.
• Figure 5: Examples of the snapshots for the velocity fields $c_1(x)$ and $c_2(x)$. Note that since $10\Delta t$ is quite small, the nearby sequential snapshots almost overlap.

Theorems & Definitions (11)

• Theorem 1
• proof
• Theorem 2
• Theorem 3: Kolmogorov Superposition Theorem lorentz1996constructive
• Theorem 4
• Remark 1
• Theorem 5
• proof
• Theorem 6
• proof