Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics

Yuxuan Liu, Jingmin Sun, Xinjie He, Griffin Pinney, Zecheng Zhang, Hayden Schaeffer

TL;DR

This work presents a new transformer-based multi-operator learning approach that fuses symbolic information to perform operator-based data prediction, i.e. non-autoregressive, and outperforms popular operator learning, computer vision, and multi-physics models, in benchmark forward prediction tasks.

Abstract

We propose PROSE-FD, a zero-shot multimodal PDE foundational model for simultaneous prediction of heterogeneous two-dimensional physical systems related to distinct fluid dynamics settings. These systems include shallow water equations and the Navier-Stokes equations with incompressible and compressible flow, regular and complex geometries, and different buoyancy settings. This work presents a new transformer-based multi-operator learning approach that fuses symbolic information to perform operator-based data prediction, i.e. non-autoregressive. By incorporating multiple modalities in the inputs, the PDE foundation model builds in a pathway for including mathematical descriptions of the physical behavior. We pre-train our foundation model on 6 parametric families of equations collected from 13 datasets, including over 60K trajectories. Our model outperforms popular operator learning, computer vision, and multi-physics models, in benchmark forward prediction tasks. We test our architecture choices with ablation studies.

PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics

TL;DR

This work presents a new transformer-based multi-operator learning approach that fuses symbolic information to perform operator-based data prediction, i.e. non-autoregressive, and outperforms popular operator learning, computer vision, and multi-physics models, in benchmark forward prediction tasks.

Abstract

We propose PROSE-FD, a zero-shot multimodal PDE foundational model for simultaneous prediction of heterogeneous two-dimensional physical systems related to distinct fluid dynamics settings. These systems include shallow water equations and the Navier-Stokes equations with incompressible and compressible flow, regular and complex geometries, and different buoyancy settings. This work presents a new transformer-based multi-operator learning approach that fuses symbolic information to perform operator-based data prediction, i.e. non-autoregressive. By incorporating multiple modalities in the inputs, the PDE foundation model builds in a pathway for including mathematical descriptions of the physical behavior. We pre-train our foundation model on 6 parametric families of equations collected from 13 datasets, including over 60K trajectories. Our model outperforms popular operator learning, computer vision, and multi-physics models, in benchmark forward prediction tasks. We test our architecture choices with ablation studies.
Paper Structure (37 sections, 9 equations, 3 figures, 3 tables)

This paper contains 37 sections, 9 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Multimodal Machine Learning (MMML)
  4. Problem Setting
  5. Model Overview
  6. Patch-based Data Encoder
  7. Equation Encoding and Fusion
  8. Patch-based Operator Decoder
  9. Experiments
  10. Experiment Setup
  11. Dataset.
  12. Evaluation Metric.
  13. Baselines and Comparisons
  14. Main Results
  15. Ablation Studies
  16. ...and 22 more sections

Figures (3)

  • Figure 1: PROSE-FD model overview. The inputs to the model are the trajectories and governing equations (in symbolic form) sampled from the datasets. The input data is patchified before being converted into a sequence of features, which are then processed with encoders and fused with processed symbolic information. The data decoder takes in fused features and generates predictions at given query locations.
  • Figure 2: Two example outputs for the PROSE-FD model.
  • Figure 3: Example outputs for the PROSE-FD model. 5 output steps for PDEArena NS-cond dataset (all three channels in equation \ref{['eq:arena_ns_c']}). Each column represents a different timestep. For this trajectory, the relative $L^2$ error is 8.74%.