The Seismic Wavefield Common Task Framework

TL;DR

The paper advocates a Seismic Wavefield Common Task Framework (CTF) to standardize evaluation of ML methods for seismic wavefield forecasting and reconstruction. It introduces three representative datasets across global, DAS, and crustal scales, along with a 12-metric evaluation scheme and a composite score to capture task-specific strengths and weaknesses. Across 17 evaluated models, many advanced methods underperform simple baselines, though recurrent architectures (notably LSTM and ODE-LSTM) show robust performance on noisy and limited-data tasks, while foundation models exhibit mixed results; the multi-metric framework reveals nuanced model capabilities and the need for task-aligned method selection. The framework aims to foster rigorous, reproducible comparisons and a living ecosystem for community-driven growth in seismic ML, with plans to expand datasets, tasks, and withheld-test evaluations to drive substantive progress.

Abstract

Seismology faces fundamental challenges in state forecasting and reconstruction (e.g., earthquake early warning and ground motion prediction) and managing the parametric variability of source locations, mechanisms, and Earth models (e.g., subsurface structure and topography effects). Addressing these with simulations is hindered by their massive scale, both in synthetic data volumes and numerical complexity, while real-data efforts are constrained by models that inadequately reflect the Earth's complexity and by sparse sensor measurements from the field. Recent machine learning (ML) efforts offer promise, but progress is obscured by a lack of proper characterization, fair reporting, and rigorous comparisons. To address this, we introduce a Common Task Framework (CTF) for ML for seismic wavefields, starting with three distinct wavefield datasets. Our CTF features a curated set of datasets at various scales (global, crustal, and local) and task-specific metrics spanning forecasting, reconstruction, and generalization under realistic constraints such as noise and limited data. Inspired by CTFs in fields like natural language processing, this framework provides a structured and rigorous foundation for head-to-head algorithm evaluation. We illustrate the evaluation procedure with scores reported for two of the datasets, showcasing the performance of various methods and foundation models for reconstructing seismic wavefields from both simulated and real-world sensor measurements. The CTF scores reveal the strengths, limitations, and suitability for specific problem classes. Our vision is to replace ad hoc comparisons with standardized evaluations on hidden test sets, raising the bar for rigor and reproducibility in scientific ML.

Figures (7)

• Figure 1: The Seismic Wavefield CTF enables the fair comparison among methods on (a) global wavefields from sparse sensor measurements, (b) dense observations of real geophysical wavefields from Distributed Acoustic Sensing, and (c) dense simulations of 3D crustal wavefields. These three datasets represent a broad range of challenging datasets encountered frequently by seismologists and present challenges for models in forecasting and state reconstruction.
• Figure 2: The Seismic Wavefield CTF scores the performance of methods on seismic wavefield datasets. (a) Data is collected and organized into matrices, which is then split into testing and training sets. RMSE errors are computed for reconstruction and short-time forecasting, while the spectral error computes the statistics of long-time forecasting (spatial or temporal). (b) Forecasting and reconstruction tasks are evaluated on noise-free, low-noise, and high-noise data. Methods are also evaluated when (c) only limited data is available and (d) for reconstruction of parametrically dependent data.
• Figure 3: Architecture of the Deep Operator Network. The target field at the evaluation point $\xi$ is approximated by the inner product of the outputs of the branch net, which takes as input the measurements $\mathbf{v}$ of the input function $v \in \mathcal{V}$ and returns a set of coefficients $\mathbf{b}(\mathbf{v})$, and the trunk net, which encodes the coordinates $\xi$ into a vector $\mathbf{t}(\xi)$.
• Figure 4: Schematic of the Sparse Identification of Nonlinear Dynamics (SINDy) algorithm from sindy, demonstrated on the Lorenz equations. The temporal evolution of the state variable and its derivative are collected in the data matrices $X$ and $\dot{X}$. The dynamical system $\dot{X} = \Theta(X)\Xi$ is then identified through sparsity promoting algorithms.
• Figure 5: Scheme of the Dynamic Mode Decomposition algorithm from kutz-dmd_book2016. The data matrix $\mathbf{X}$ is constructed by stacking the snapshots in columns. The SVD of the data matrix is computed, and the dynamical matrix is fitted to the data. This allows us to compute the state of the system for future time instances.