A Multimodal Conditional Mixture Model with Distribution-Level Physics Priors

TL;DR

The paper tackles the challenge of modeling intrinsically multimodal conditional distributions in physics-grounded systems by introducing a physics-informed Mixture Density Network (MDN) framework. The method represents $p(u|x)$ as a mixture of Gaussians with input-dependent parameters, and enforces physical consistency through component-specific regularization terms weighted by the mixture probabilities, enabling conditional modeling that respects governing equations. Key innovations include a class-conditioned extension to align mixture components with physical regimes, and physics priors such as monotonicity and PDE residual-based losses, demonstrated across bifurcation, multiscale SPDEs, shock physics, and reaction–diffusion problems, with competitive results to conditional flow matching (CFM). The work offers interpretable, likelihood-based uncertainty quantification via closed-form moments and remains computationally efficient, though it notes limitations like the Gaussian assumption and absence of explicit epistemic uncertainty modeling, pointing to future extensions for richer conditionals and uncertainty characterizations.

Abstract

Many scientific and engineering systems exhibit intrinsically multimodal behavior arising from latent regime switching and non-unique physical mechanisms. In such settings, learning the full conditional distribution of admissible outcomes in a physically consistent and interpretable manner remains a challenge. While recent advances in machine learning have enabled powerful multimodal generative modeling, their integration with physics-constrained scientific modeling remains nontrivial, particularly when physical structure must be preserved or data are limited. This work develops a physics-informed multimodal conditional modeling framework based on mixture density representations. Mixture density networks (MDNs) provide an explicit and interpretable parameterization of multimodal conditional distributions. Physical knowledge is embedded through component-specific regularization terms that penalize violations of governing equations or physical laws. This formulation naturally accommodates non-uniqueness and stochasticity while remaining computationally efficient and amenable to conditioning on contextual inputs. The proposed framework is evaluated across a range of scientific problems in which multimodality arises from intrinsic physical mechanisms rather than observational noise, including bifurcation phenomena in nonlinear dynamical systems, stochastic partial differential equations, and atomistic-scale shock dynamics. In addition, the proposed method is compared with a conditional flow matching (CFM) model, a representative state-of-the-art generative modeling approach, demonstrating that MDNs can achieve competitive performance while offering a simpler and more interpretable formulation.

Figures (9)

• Figure 1: Illustrations of the MDN architecture and the proposed multimodal conditional modeling framework.
• Figure 2: MDN training for probabilistic modeling of bifurcated response branches. (\ref{['fig:bifurcation_main_a']}) Loss history over $20{,}000$ ADAM iterations. (\ref{['fig:bifurcation_main_b']}) Predicted component means $\mu_m$ and standard deviations $\sigma_m$ and (\ref{['fig:bifurcation_main_c']}) predicted mixing coefficients $\pi_m$. (\ref{['fig:bifurcation_main_d']}) $5{,}000$ samples drawn from the trained MDN model.
• Figure 3: Comparison between MDN and CFM both after $20{,}000$ ADAM iterations. The number of model parameters of MDN is $457$, where CFM consists of a comparable size of $521$ parameters. (\ref{['fig:bifurcation_sampling_a']}) Sample distribution of CFM model output conditioned on $5{,}000$ equispaced $\lambda$ values between $-3$ and $3$. Distribution of true distribution, MDN, and $5{,}000$ CFM samples, conditioned on (\ref{['fig:bifurcation_sampling_b']}) $\lambda=-0.8$, (\ref{['fig:bifurcation_sampling_c']}) $\lambda=0$, and (\ref{['fig:bifurcation_sampling_d']}) $\lambda=0.8$.
• Figure 4: Results of training MDNs with two mixture components on SPDE data. SPDE data are randomly subsampled into size $10{,}000$ out of $10{,}000{,}000$ data points for visibility. (\ref{['fig:spde_sampling_a']}) $1{,}000$ samples from MDN model conditioned on $u_1=5$, and (\ref{['fig:spde_sampling_b']}) corresponding PDF of MDN and true distribution. (\ref{['fig:spde_sampling_c']}) $1{,}000$ samples from MDN model conditioned on $u_1=12$, and (\ref{['fig:spde_sampling_d']}) corresponding PDF of MDN and true distribution.
• Figure 5: Shock Hugoniots of single-crystal 3C--SiC data along the $[001]$ orientation, digitized and reproduced from li2017shockbranicio2018planepasparakis2026physics.

Theorems & Definitions (2)

• Remark 2.1
• Remark 2.2