Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks

Abstract

Physics-informed neural networks (PINNs) have achieved notable success in modeling dynamical systems governed by partial differential equations (PDEs). To avoid computationally expensive retraining under new physical conditions, parameterized PINNs (P$^2$INNs) commonly adapt pre-trained operators using singular value decomposition (SVD) for out-of-distribution (OOD) regimes. However, SVD-based fine-tuning often suffers from rigid subspace locking and truncation of important high-frequency spectral modes, limiting its ability to capture complex physical transitions. While parameter-efficient fine-tuning (PEFT) methods appear to be promising alternatives, applying conventional adapters such as LoRA to P$^2$INNs introduces a severe Pareto trade-off, as additive updates increase parameter overhead and disrupt the structured physical manifolds inherent in operator representations. To address these limitations, we propose Manifold-Orthogonal Dual-spectrum Extrapolation (MODE), a lightweight micro-architecture designed for physics operator adaptation. MODE decomposes physical evolution into complementary mechanisms including principal-spectrum dense mixing that enables cross-modal energy transfer within frozen orthogonal bases, residual-spectrum awakening that activates high-frequency spectral components through a single trainable scalar, and affine Galilean unlocking that explicitly isolates spatial translation dynamics. Experiments on challenging PDE benchmarks including the 1D Convection--Diffusion--Reaction equation and the 2D Helmholtz equation demonstrate that MODE achieves strong out-of-distribution generalization while preserving the minimal parameter complexity of native SVD and outperforming existing PEFT-based baselines.

Figures (5)

• Figure 1: Comparison of different fine-tuning approaches for P$^2$INNs. (a) Native Singular Value Decomposition (SVD), (b) Weight-Decomposed Adaptation (DoRA), and (c) our proposed Manifold-Orthogonal Dual-spectrum Extrapolation (MODE). (■: frozen parameters; ■: tunable parameters; ■: zeros.)
• Figure 2: Structural limitations of SVD-based fine-tuning in P$^2$INNs. (a) Degrees of freedom comparison between SVD and LoRA. (b) Approximation error of SVD versus optimal rank-$r$ projection. (c) PDE scenario comparing fixed-basis and free-basis projections.
• Figure 3: Subspace rotation failure of P$^2$INN-SVD when generalizing to the OOD convection equation $u_t + 30\,u_x = 0$.
• Figure 4: P$^2$INNs with MODE modulation. Each intermediate layer of the manifold decoder is replaced by a dual-path MODE-adapted block: (i) the Frozen Full-Rank Path scales the frozen weight output $\mathbf{W}_{0,l}\mathbf{h}^{(l-1)}$ by the trainable scalar $\tau_l$; (ii) the Ultra-Low-Rank Orthogonal Path mixes cross-modal energy via the dense core $\mathbf{\Phi}_l$ within the compact $k$-dimensional subspace. Only $\{\mathbf{\Phi}_l, \tau_l, \Delta\mathbf{b}_l\}$ are trainable; all other components are strictly frozen.
• Figure 5: Pre-training loss curves on the 1D CDR equations for 10 different parameter values. (a) Convection-Diffusion Equations; (b) Diffusion-Reaction Equations; (c) Convection-Diffusion-Reaction Equations.

Theorems & Definitions (6)

• Remark 1: Pathology of Native SVF
• Remark 2: Pathology of Additive Low-Rank & Spectral Variants
• Theorem 1: Zero-Degradation Exact Recovery
• proof
• Theorem 2: Asymptotic Pareto Spatial Complexity
• proof