Parameter Efficient Fine-Tuning for Deep Learning-Based Full-Waveform Inversion

TL;DR

This work tackles the generalization challenge in seismic full waveform inversion by proposing a task-agnostic foundational model pretrained on diverse OpenFWI data. By applying parameter-efficient fine-tuning (PEFT), notably LoRA, it demonstrates that a frozen or lightly updated backbone can adapt to multiple FWI tasks with substantially reduced memory and computation while maintaining or improving performance, especially on out-of-distribution data and in low-data regimes. The approach shows that full fine-tuning of the foundation (FFT-PFM) outperforms task-specific models on in-distribution data, but LoRA-PFM provides superior robustness and generalization on OOD tasks, with considerably lower memory footprints. These findings suggest that PFMs combined with PEFT offer a scalable, generalizable, and resource-efficient pathway for DL-based FWI, with potential applicability to broader geophysical challenges beyond FWI.

Abstract

Seismic full waveform inversion (FWI) has seen promising advancements through deep learning. Existing approaches typically focus on task-specific models trained and evaluated in isolation that lead to limited generalization across different geological scenarios. In this work we introduce a task-agnostic foundational model for FWI that captures general features across tasks. We first demonstrate that full fine-tuning of this foundational model outperforms task-specific models built from scratch by delivering superior performance across multiple benchmarks. Building upon this we employ parameter-efficient fine-tuning (PEFT) to further reduce computational overhead. By fine-tuning only a small fraction of the model parameters PEFT achieves comparable results to full fine-tuning while significantly lowering memory and computational requirements. Additionally, PEFT excels in out-of-distribution tasks where it outperforms both full fine-tuning and task-specific models. These findings establish the value of foundational modeling for FWI and highlight PEFT as an effective strategy for efficient and scalable adaptation across diverse tasks.

Figures (15)

• Figure 1: Comparison of our proposed approach with the traditional fine-tuning method. The left-hand side figure shows the traditional task-specific model and traditional fine-tuning method. The right-hand side figure represents the pertaining of the foundational model and subsequent two types of fine-tuning methods: Full fine-tuning(FFT-PFM) and LoRA-PFM.
• Figure 2: Performance improvement of the pretrained foundational model trained on six datasets from OpenFWI is analyzed over the baseline in terms of three accuracy metrics: MAE, RMSE, and SSIM. The abbreviation are FVA: FlatVel A, CVA: CurveVel A, CVB: CurveVel B, FFA: FlatFault A, FFB: FlatFault B, CFA: CurveFault A
• Figure 3: Performance improvement of the PFM, when fine-tuning on four datasets over the baseline in terms of MAE, RMSE, and SSIM.
• Figure 4: Comparison of full fine-tuning and LoRA-PFM over FlatVel B, CurveFault B, Style A, and Style B. Randomly selected two samples from the same dataset and (a), (b), (c), and (d) in the diagram represents FlatVel B, CurveFault B, Style A and Style B respectively.
• Figure 5: Generalization improvement for fine-tuning with 10%, 25%, 50%, 75% and 100% of training dataset and test with CurveFault B (blue), Style A (red), Style B (green). Evaluation was based on metric score, MAE, RMSE and SSIM. For MAE and RMSE, a lower value indicates better performance, while the reverse is true for SSIM.