RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data

TL;DR

RealPDEBench addresses the scarcity of real-world data in scientific ML by pairing real measurements with matched simulations across five physical scenarios. It defines three prediction tasks—real-world training, simulated training, and simulated pretraining with real-world finetuning—and employs a unified nine-metric framework to evaluate data- and physics-driven performance, including autoregressive horizons. The study reveals a noticeable gap between real and simulated data but demonstrates that simulated pretraining can improve real-world accuracy and convergence, guiding more robust sim-to-real deployment. The benchmark, datasets, and code enable systematic evaluation of PDE-focused foundation models and neural operators, advancing practical scientific ML in real-world settings.

Abstract

Predicting the evolution of complex physical systems remains a central problem in science and engineering. Despite rapid progress in scientific Machine Learning (ML) models, a critical bottleneck is the lack of expensive real-world data, resulting in most current models being trained and validated on simulated data. Beyond limiting the development and evaluation of scientific ML, this gap also hinders research into essential tasks such as sim-to-real transfer. We introduce RealPDEBench, the first benchmark for scientific ML that integrates real-world measurements with paired numerical simulations. RealPDEBench consists of five datasets, three tasks, eight metrics, and ten baselines. We first present five real-world measured datasets with paired simulated datasets across different complex physical systems. We further define three tasks, which allow comparisons between real-world and simulated data, and facilitate the development of methods to bridge the two. Moreover, we design eight evaluation metrics, spanning data-oriented and physics-oriented metrics, and finally benchmark ten representative baselines, including state-of-the-art models, pretrained PDE foundation models, and a traditional method. Experiments reveal significant discrepancies between simulated and real-world data, while showing that pretraining with simulated data consistently improves both accuracy and convergence. In this work, we hope to provide insights from real-world data, advancing scientific ML toward bridging the sim-to-real gap and real-world deployment. Our benchmark, datasets, and instructions are available at https://realpdebench.github.io/.

Figures (21)

• Figure 1: Five scenarios in RealPDEBench with the corresponding real-world and simulated data. It demonstrates the differences between real-world and simulated data, such as the different modalities, measurement noise, and numerical errors. These discrepancies motivate and call for our proposed benchmark, RealPDEBench, to systematically collect data and conduct experimental analysis.
• Figure 2: Photos of real-world data collection. (a) Water tunnel after laser irradiation. (b) Particle imaging photos taken by the camera. (c) Motion control equipment. (d) Swirl combustor equipment.
• Figure 3: (a) Frequency errors of baselines (statistics of 10 values) on real-world vs. simulated data from Controlled Cylinder. (b) Validation RMSE curves of real-world finetuning on Combustion, with crosses marking the best RMSE of real-world training. The x-axis shows the percentage of update iterations. (c) RMSE under 1, 2, 3, 5, and 10 rounds of autoregressive evaluation on Cylinder.
• Figure 4: Trade-off of RMSE and Frequency Error. Distant points are linked with dashed lines. We omit the points where the DMD error is far from the center to make most of the points clearer.
• Figure 5: (a) MVPEs of real-world finetuning under 10-round autoregressive evaluation on Cylinder. (b) MVP of U-Net's 10-round autoregressive prediction on Cylinder.