PhysFormer: A Physics-Embedded Generative Model for Physically Self-Consistent Spectral Synthesis

TL;DR

PhysFormer is proposed, a generative modeling framework that is self-consistent at both the data and physical levels, providing a physically consistent generative modeling paradigm for complex systems involving unknown or unobservable physical quantities.

Abstract

In scientific and engineering domains, modeling high-dimensional complex systems governed by partial differential equations (PDEs) remains challenging in terms of physical consistency and numerical stability. However, existing approaches, such as physics-informed neural networks (PINNs), typically rely on known physical fields or coefficients and enforce physical constraints via external loss functions, which can lead to training instability and make it difficult to handle high-dimensional or unobservable scenarios. To this end, we propose PhysFormer, a generative modeling framework that is self-consistent at both the data and physical levels. PhysFormer leverages a low-dimensional, physically interpretable latent space to learn key physical quantities directly from data without requiring known high-dimensional physical field parameters, and embeds the physical process of radiative flux generation within the network to ensure the physical consistency of the generated spectra. In high-dimensional, degenerate inversion tasks, PhysFormer constrains generation within physical limits and enhances spectral fidelity and inversion stability under varying signal-to-noise ratios (SNRs). More broadly, this approach shifts the physical processes from external loss functions into the generative mechanism itself, providing a physically consistent generative modeling paradigm for complex systems involving unknown or unobservable physical quantities.

Figures (5)

• Figure 1: The architecture of PhysFormer. (a) Physical generation module that embeds physical processes directly into the network architecture. (b) Physically consistent spectral autoencoder. (c) Bottleneck architecture for generating a low-dimensional physical latent space. (d) Mapping network from stellar fundamental parameters to the physical latent space. (e) Latent Space Expansion Module. (f) PhysFormer decoder, used to generate key physical quantities required for subsequent physical computations. (g) Radiative Intensity Generation Module. (h) Transformer Block structure.
• Figure 2: Visualization of Spectra Generated by PhysFormer. Left: comparison between PhysFormer-generated spectra and ground truth. Middle: comparison among PhysFormer, Symmetric Autoencoder, and SPECULATOR. Top right: median spectral error across wavelengths. Bottom right: visualization of the learned low-dimensional physical latent space.
• Figure 3: Residual Distributions of RTE and ET on the Test Set. Top row: Residual distributions of the Radiative Transfer Equation (RTE) and Energy Theorem (ET) for the three models: PhysFormer, PhysFormer without Bottleneck, and MLP Decoder. Bottom row: Absolute residual distributions of RTE and ET for the first 100 wavelength points for PhysFormer.
• Figure 4: Loss landscape in $T_{\mathrm{eff}}$--$\log g$ space for a single observed spectrum. Color indicates the spectrum reconstruction MSE. The red cross marks the recovered parameter. The elongated and tilted contours indicate a strong degeneracy between the two parameters.
• Figure 5: Example of Parameter Inversion Results and Error Distributions. Top: Example spectra generated from the inverted parameters, comparing the output of PhysFormer with PhysFormer w/o Physics Process. Bottom: Distributions of absolute errors in the inversion of four-dimensional stellar parameters under different signal-to-noise ratio (SNR) conditions for the two models.