Efficient physics-informed neural networks using hash encoding

TL;DR

This work tackles the high training cost of physics-informed neural networks (PINNs) by introducing multi-resolution hash encoding to create locally-aware coordinate embeddings, accelerating convergence on PDE problems. A key challenge is that hash encoding induces derivative discontinuities, which the authors address by replacing automatic differentiation with finite-difference derivatives, and by providing guidance on robust hash-hyperparameter ranges. The method is validated on Burgers, Helmholtz, and Navier–Stokes equations, achieving substantial speedups (roughly an order of magnitude) while maintaining accuracy. The results suggest that hash-encoded PINNs can offer practical, near-real-time training for PDE solvers, with avenues for further improvement via higher-order interpolation and refined hyperparameter tuning.

Abstract

Physics-informed neural networks (PINNs) have attracted a lot of attention in scientific computing as their functional representation of partial differential equation (PDE) solutions offers flexibility and accuracy features. However, their training cost has limited their practical use as a real alternative to classic numerical methods. Thus, we propose to incorporate multi-resolution hash encoding into PINNs to improve the training efficiency, as such encoding offers a locally-aware (at multi resolution) coordinate inputs to the neural network. Borrowed from the neural representation field community (NeRF), we investigate the robustness of calculating the derivatives of such hash encoded neural networks with respect to the input coordinates, which is often needed by the PINN loss terms. We propose to replace the automatic differentiation with finite-difference calculations of the derivatives to address the discontinuous nature of such derivatives. We also share the appropriate ranges for the hash encoding hyperparameters to obtain robust derivatives. We test the proposed method on three problems, including Burgers equation, Helmholtz equation, and Navier-Stokes equation. The proposed method admits about a 10-fold improvement in efficiency over the vanilla PINN implementation.

Figures (6)

• Figure 1: The diagram of the hash encoding, where different colors denote the different scales (resolution) and corresponding embedding vectors.
• Figure 2: Illustration of the accuracy of the first- and second-order derivatives calculation by the AD method. We use this NN to fit $f=sin(x)$ with a multi-resolution hash encoding and visualize its first- and second-order derivatives for a hash table size of 10 in (a), and also visualize the derivatives with hash table sizes of 8 and 4 in (b) and (c), respectively.
• Figure 3: Illustration of the accuracy of the first- and second-order derivatives calculation by the FD method. We use an NN to fit $f=sin(x)$ with the multi-resolution hash encoding and visualize its first- and second-order derivatives for a hash table size of 10 in (a), and also visualize the derivatives with hash table sizes of 8 and 4 in (b) and (c), respectively.
• Figure 4: a) The histories of convergence and testing data errors for the Burgers equation tests, and b) the prediction of PINN with hash encoding and the numerical reference solutions.
• Figure 5: The histories of convergence and testing data errors for the Helmholtz equation tests, and b) the prediction of PINN with hash encoding and the numerical reference solutions.