Predicting Dynamical Systems across Environments via Diffusive Model Weight Generation
Ruikun Li, Huandong Wang, Jingtao Ding, Yuan Yuan, Qingmin Liao, Yong Li
TL;DR
This work addresses the challenge of predicting dynamical systems across diverse environments by treating neural network weights as a data modality and explicitly modeling their conditional distribution with environments. It introduces EnvAd-Diff, a pipeline that builds a domain-adaptive model zoo, encodes environment-specific weights as graphs via a weight-graph VAE, and learns a latent diffusion model conditioned on environment to generate zero-shot, environment-tailored predictors. A physics-informed prompter distinguishes unseen environments when prior environmental knowledge is unavailable, enabling effective weight generation even without data from the target environment. Across multiple PDE systems and a real-world ERA5 dataset, EnvAd-Diff delivers superior cross-environment generalization with far fewer parameters than foundation models, and provides explanations through weight-environment visualizations and LV-type symbolic structure. The work advances cross-environment scientific machine learning by leveraging structured weight representations and diffusion-based generative modeling, with practical implications for zero-shot deployment of dynamical predictors.
Abstract
Data-driven methods offer an effective equation-free solution for predicting physical dynamics. However, the same physical system can exhibit significantly different dynamic behaviors in various environments. This causes prediction functions trained for specific environments to fail when transferred to unseen environments. Therefore, cross-environment prediction requires modeling the dynamic functions of different environments. In this work, we propose a model weight generation method, \texttt{EnvAd-Diff}. \texttt{EnvAd-Diff} operates in the weight space of the dynamic function, generating suitable weights from scratch based on environmental condition for zero-shot prediction. Specifically, we first train expert prediction functions on dynamic trajectories from a limited set of visible environments to create a model zoo, thereby constructing sample pairs of prediction function weights and their corresponding environments. Subsequently, we train a latent space diffusion model conditioned on the environment to model the joint distribution of weights and environments. Considering the lack of environmental prior knowledge in real-world scenarios, we propose a physics-informed surrogate label to distinguish different environments. Generalization experiments across multiple systems demonstrate that a 1M parameter prediction function generated by \texttt{EnvAd-Diff} outperforms a pre-trained 500M parameter foundation model.