Multi-Objective Loss Balancing in Physics-Informed Neural Networks for Fluid Flow Applications

TL;DR

The paper addresses multi-objective loss balancing in PINNs for Navier–Stokes fluid flows by introducing trainable SiLU activations and learnable B-spline bases within Kolmogorov–Arnol'd networks. It systematically evaluates how activation design interacts with loss-balancing strategies (RBA, SA, LRA, GradNorm) across 2D lid-driven cavity, plane Poiseuille, and BFS slip/no-slip flows, comparing against fixed Tanh baselines and OpenFOAM references. The key finding is that learnable activations generally improve RMSE (7.4%–95.2% reductions) with most schemes; LRA often yields the largest gains, though GradNorm can become unstable with these activations. The study also highlights trade-offs: while accuracy improves, the B-spline+SiLU networks incur higher parameter counts and runtime, with RBA offering a favorable efficiency-accuracy balance and GradNorm proving unreliable in some configurations.

Abstract

Physics-Informed Neural Networks (PINNs) have emerged as a promising machine learning approach for solving partial differential equations (PDEs). However, PINNs face significant challenges in balancing multi-objective losses, as multiple competing loss terms such as physics residuals, boundary conditions, and initial conditions must be appropriately weighted. While various loss balancing schemes have been proposed, they have been implemented within neural network architectures with fixed activation functions, and their effectiveness has been assessed using simpler PDEs. We hypothesize that the effectiveness of loss balancing schemes depends not only on the balancing strategy itself, but also on the loss function design and the neural network's inherent function approximation capabilities, which are influenced by the choice of activation function. In this paper, we extend existing solutions by incorporating trainable activation functions within the neural network architecture and evaluate the proposed approach on complex fluid flow applications modeled by the Navier-Stokes equations. Our evaluation across diverse Navier-Stokes problems demonstrates that this proposed solution achieves root mean square error (RMSE) improvements ranging from 7.4% to 95.2% across different scenarios. These findings highlight the importance of carefully designing the loss function and selecting activation functions for effective loss balancing.

Figures (6)

• Figure 1: The diagram illustrates two neural network architectures: the Tanh activation function is applied in a traditional multi-layer perceptron (MLP), while SiLU+B-Spline is utilized in Kolmogorov-Arnold Networks (KAN). The predictions of these networks are fed to a PINN. The PINN's loss function includes Navier-Stokes loss ($f_{\text{NS}}$), $m$ boundary condition losses ($\mathcal{L}_{\text{BC}_1} \dots \mathcal{L}_{\text{BC}_m}$), and an initial condition loss ($\mathcal{L}_{\text{IC}}$). As a result, the total loss becomes a multi-objective optimization problem, expressed as $\mathcal{L}_{\text{PINN}} = \textcolor{#82B366}{\lambda_1}f_{\text{NS}} + \textcolor{#B85450}{\lambda_{BC_1}} \mathcal{L}_{\text{BC}_1} + \dots + \textcolor{#B85450}{\lambda_{BC_m}} \mathcal{L}_{\text{BC}_m} + \textcolor{#9673A6}{\lambda_n} \mathcal{L}_{\text{IC}}$. The weighting factors, $\textcolor{#82B366}{\lambda_1}$, $\textcolor{#B85450}{\lambda_{BC_1}} \dots \textcolor{#B85450}{\lambda_{BC_m}}$, and $\textcolor{#9673A6}{\lambda_n}$ are also treated as trainable parameters, and their optimal values are learned during training, along with the neural network weights.
• Figure 2: Training loss convergence curves for the loss balancing schemes across the selected fluid flow problems using Tanh (top) and B-spline+SiLU (bottom) activation functions.
• Figure 3: Contour plots for the solution of the Cavity problem, showing velocity components ($u_x$ and $v_y$) and pressure ($p$) values at the end of the simulation (steady state). The PINN model yields the best results with LRA, as shown in Table \ref{['tab:performance']}.
• Figure 4: Contour plots for the solution of the Poiseuille problem, showing velocity components ($u_x$ and $v_y$) and pressure ($p$) values at the steady state. The PINN model shows results with the fixed heuristic weighting with 0.1, 2, 2 for $\lambda_{phy}$, $\lambda_{bc}$, and $\lambda_{ic}$, respectively, which gives the best results as shown in Table \ref{['tab:performance']}.
• Figure 5: Contour plots for the solution of the BFS-Slip problem, showing velocity components ($u_x$ and $v_y$) and pressure ($p$) values at the steady state. PINN model predictions are drawn with the SA method.