FMint-SDE: A Multimodal Foundation Model for Accelerating Numerical Simulation of SDEs via Error Correction
Jiaxin Yuan, Haizhao Yang, Maria Cameron
TL;DR
FMint-SDE addresses the computational bottleneck in simulating stochastic differential equations by uniting traditional coarse-time-step solvers with a multimodal, transformer-based foundation model that learns error corrections from in-context demonstrations. The decoder-only architecture processes numerical trajectories and optional textual prompts to predict correction terms that reconcile coarse simulations with fine solutions, achieving near-fine accuracy at coarse-run timescales. Pretrained on four SDE families and evaluated across twelve benchmarks, FMint-SDE demonstrates strong zero-shot transfer and robust improvements under limited fine-tuning, with a roll-out scheme enabling long-horizon simulations. The approach offers a scalable, general-purpose tool for high-fidelity dynamical-system simulations with potential impact across molecular dynamics, physics-based engineering, and quantitative finance. All results are framed with $dX(t) = b(X_t,t) \, dt + \sigma(X_t,t) \, dW_t$, highlighting the method’s foundation in stochastic calculus and its practical translation to efficient simulations.
Abstract
Fast and accurate simulation of dynamical systems is a fundamental challenge across scientific and engineering domains. Traditional numerical integrators often face a trade-off between accuracy and computational efficiency, while existing neural network-based approaches typically require training a separate model for each case. To overcome these limitations, we introduce a novel multi-modal foundation model for large-scale simulations of differential equations: FMint-SDE (Foundation Model based on Initialization for stochastic differential equations). Based on a decoder-only transformer with in-context learning, FMint-SDE leverages numerical and textual modalities to learn a universal error-correction scheme. It is trained using prompted sequences of coarse solutions generated by conventional solvers, enabling broad generalization across diverse systems. We evaluate our models on a suite of challenging SDE benchmarks spanning applications in molecular dynamics, mechanical systems, finance, and biology. Experimental results show that our approach achieves a superior accuracy-efficiency tradeoff compared to classical solvers, underscoring the potential of FMint-SDE as a general-purpose simulation tool for dynamical systems.