Interpolating neural network: A novel unification of machine learning and interpolation theory

TL;DR

An interpolating neural network (INN) is introduced, grounded in interpolation theory and tensor decomposition, to realize Engineering Software 2.0 by advancing data training, partial differential equation solving, and parameter calibration.

Abstract

Artificial intelligence (AI) has revolutionized software development, shifting from task-specific codes (Software 1.0) to neural network-based approaches (Software 2.0). However, applying this transition in engineering software presents challenges, including low surrogate model accuracy, the curse of dimensionality in inverse design, and rising complexity in physical simulations. We introduce an interpolating neural network (INN), grounded in interpolation theory and tensor decomposition, to realize Engineering Software 2.0 by advancing data training, partial differential equation solving, and parameter calibration. INN offers orders of magnitude fewer trainable/solvable parameters for comparable model accuracy than traditional multi-layer perceptron (MLP) or physics-informed neural networks (PINN). Demonstrated in metal additive manufacturing, INN rapidly constructs an accurate surrogate model of Laser Powder Bed Fusion (L-PBF) heat transfer simulation, achieving sub-10-micrometer resolution for a 10 mm path in under 15 minutes on a single GPU. This makes a transformative step forward across all domains essential to engineering software.

Figures (4)

• Figure 1: Overview of Engineering Software 2.0 enabled with Interpolating Neural Network (INN). The INN box illustrates the graph representation of the input domain discretized with an arbitrary Euclidean mesh (left) and a regular mesh as a special case (right). Node and edge features are given from the discretized mesh. After $Q$-step message passing, each node $j$ will store an interpolation function $N^{[Q](j)}(\bm{x})$. Finally, the readout operation sums the product of the interpolation functions and nodal values $\bm{u}^{(j)}$. The superscripts with square brackets [] and parentheses () denote the message passing step and graph node index, respectively. The interpolation operator $\mathcal{J}$ denotes that $\mathcal{J}\bm{u}(\bm{x})$ is a function that interpolates discrete values of $\bm{u}^{(j)}$. The INN trainer employs data-driven loss functions (e.g., mean squared error loss for regression) while the INN solver adopts a residual loss of a partial differential equation (PDE). A trained/solved INN model can then be employed as a forward model of a calibrator to solve an inverse problem.
• Figure 2: Illustration of a 1D input domain discretized with 5 segments and 6 nodes.
• Figure 3: Benchmarks for the INN solver (a) and the INN trainer (b). In (a), a 3D-space, time, and 4D-parameter heat transfer equation is solved to model a 10 mm single-track laser powder bed fusion (L-PBF) metal AM. Detailed problem definition and explanation can be found in Section \ref{['sec:applications']}. We compare the single-scale finite difference method (FDM) solver liao2023efficient, the variational multiscale FEM solver leonor2024go, and the INN solver with CP decomposition and Q=2. The data points for the first two methods with dashed marker edges are estimated, while that of INN with a solid marker edge is computed. In (b), a standard data training (regression) problem is studied. Each trainer stops when the training loss hits the stopping criteria (MSE:4e-4). We set the same optimization condition: ADAM optimizer; learning rate: 1e-3; batch size: 128. The number of neurons per hidden layer is denoted for each MLP data point while the number of segments and modes are denoted for each INN data point. INN with CP decomposition and Q=2 is used. In (c), a 1D Poisson's equation is solved with PINN and INN. PINN is made of a 1-layer MLP with a varying number of neurons. Randomly selected 10k collocation points are used to compute the PDE loss. With a batch size of 128, 10k epochs are trained using an ADAM optimizer with a learning rate of 1e-1. FEM with linear elements is equivalent to an INN with Q=1, s=1. C-HiDeNNs with s=3,P=3 and with s=5,P=5 are equivalent to INNs with Q=2, s=3, P=3, and with Q=2, s=5, P=5, respectively. All solvers and trainers are graphics processing units (GPU) optimized with the JAX library jax2018github.
• Figure 4: A summary of engineering applications of INNs to additive manufacturing (AM) problems. First, a data-driven real-time online monitoring and feedback control tool is formulated with an INN that uses only 18% training parameters of MLP and is 18-31 times faster than MLP. The calibration problem develops a reduced-order model of laser powder bed fusion (L-PBF) AM to calibrate the heat source parameters from experimental data. Finally, the INN solver solves a space-time-parameter heat transfer equation, resulting in a significant storage reduction and faster simulations.