Towards Physics-Guided Foundation Models
Majid Farhadloo, Arun Sharma, Mingzhou Yang, Bharat Jayaprakash, William Northrop, Shashi Shekhar
TL;DR
This work addresses the limitations of purely data-driven foundation models in physics-rich domains, notably poor out-of-distribution generalization and violations of fundamental physical principles. It introduces Physics-Guided Foundation Models (PGFM), formalizes the concept, and surveys methods to integrate broad-domain physical knowledge into pretraining and downstream fine-tuning. Key contributions include clarifying wide versus narrow physics knowledge, proposing physics-constrained learning and architecture-level physics integration, and illustrating benefits with velocity-jerk examples and domain datasets. The authors argue that PGFMs can improve reliability, robustness, and domain trust across geoscience, healthcare, and engineering tasks, and they outline future directions such as retrieval-augmented updates and multi-modal physics integration.
Abstract
Traditional foundation models are pre-trained on broad datasets to reduce the training resources (e.g., time, energy, labeled samples) needed for fine-tuning a wide range of downstream tasks. However, traditional foundation models struggle with out-of-distribution prediction and can produce outputs that are unrealistic and physically infeasible. We propose the notation of physics-guided foundation models (PGFM), that is, foundation models integrated with broad or general domain (e.g., scientific) physical knowledge applicable to a wide range of downstream tasks.