Physics-informed Neural Networks for Encoding Dynamics in Real Physical Systems

TL;DR

The work investigates physics-informed neural networks (PINNs) as predictive models that encode governing differential equations and assesses their performance on two physical systems: a simple nonlinear pendulum and 2D heat diffusion on a metal block. It demonstrates substantial accuracy gains for PINNs over uninformed neural networks in the pendulum problem, both with ideal data (up to ~18x improvement) and real experimental data (around ~9x improvement), while also highlighting optimization and data-denoising strategies that aid PINN training. For the heat diffusion case, the study uncovers significant challenges in large-domain optimization, showing that data denoising, frame-size reduction, and using LBFGS can improve results, but overall PINNs and NNs struggle here due to the problem's scale and noise. The work also explores deploying physics-informed models on edge hardware (PYNQ-Z1 FPGA) and identifies critical issues in time-coherent sensing and spatial data alignment that must be addressed for robust real-time inference and potential model predictive control. Together, these findings clarify both the promise and the practical hurdles of integrating physics knowledge into ML for real physical systems and guide future development toward scalable, hardware-aware, physics-informed inference.

Abstract

This dissertation investigates physics-informed neural networks (PINNs) as candidate models for encoding governing equations, and assesses their performance on experimental data from two different systems. The first system is a simple nonlinear pendulum, and the second is 2D heat diffusion across the surface of a metal block. We show that for the pendulum system the PINNs outperformed equivalent uninformed neural networks (NNs) in the ideal data case, with accuracy improvements of 18x and 6x for 10 linearly-spaced and 10 uniformly-distributed random training points respectively. In similar test cases with real data collected from an experiment, PINNs outperformed NNs with 9.3x and 9.1x accuracy improvements for 67 linearly-spaced and uniformly-distributed random points respectively. For the 2D heat diffusion, we show that both PINNs and NNs do not fare very well in reconstructing the heating regime due to difficulties in optimizing the network parameters over a large domain in both time and space. We highlight that data denoising and smoothing, reducing the size of the optimization problem, and using LBFGS as the optimizer are all ways to improve the accuracy of the predicted solution for both PINNs and NNs. Additionally, we address the viability of deploying physics-informed models within physical systems, and we choose FPGAs as the compute substrate for deployment. In light of this, we perform our experiments using a PYNQ-Z1 FPGA and identify issues related to time-coherent sensing and spatial data alignment. We discuss the insights gained from this work and list future work items based on the proposed architecture for the system that our methods work to develop.

Figures (46)

• Figure 1: Example of a PINN architecture based on the 2D heat equation using trainable parameters $\theta_n$. The left dashed box shows the neural network which predicts the value of $u$ given the training points to produce the data loss term. The right dashed box shows the PDE residual corresponding to the heat equation, composed from the differential terms. The differential terms are obtained using automatic differentiation. The PDE residual forms the physics loss, which is the distinguishing component of PINNs.
• Figure 2: Illustrative diagram of a pendulum system.
• Figure 3: Numerical solution of a pendulum system generated using Equations \ref{['eq:ang_veloc_update']} and \ref{['eq:ang_displ_update']}.
• Figure 4: Numerical solution of a pendulum system generated using Equations \ref{['eq:ang_veloc_update_air_resist']} and \ref{['eq:ang_displ_update']}, taking air resistance into consideration. A more realistic solution would consider a smaller amount of damping over a longer interval, but for our purposes this solution is sufficient.
• Figure 5: PINN predictions on the synthetic data pendulum given 150 training points. PINN architecture: 3 FC hidden layers with 32 neurons each. RMSE = 0.0068 as the PINN has no trouble fitting the data given a perfect setup.